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Casandra  E.  O' Neal  I  ■  Scott  S, 


For  more  than  a  decade,  Crane  Army  Ammunition  Activity  (CAAA)  has  pro¬ 
duced  and  provided  conventional  munitions  to  the  warfighters,  supporting 
them  in  winning  battles  in  Iraq  and  Afghanistan.  Its  senior  leaders  continually 
strive  to  improve  its  logistics  processes.  Due  to  a  rapidly  changing  environ¬ 
ment,  this  Army  Working  Capital  Fund  organization  began  a  dramatic  trans¬ 
formation  by  replacing  its  operational  processes,  moving  from  the  proven  concepts 
of  logistics  to  the  more  robust  and  flexible  approaches  of  Supply  Chain  Management 


(SCM).  Along  the  way,  it  had  to  renovate  the  way  it  measured  its  organizational 
performance.  CAAA  leaders  now  use  this  new  balanced  scorecard  based  upon 
SCMJo  make  important  organizational  decisions. 


Supply  Chain  Management 

SCM  has  become  a  vital  tool  used  in  today's  global 
economy  to  increase  success  and  efficiency  in  the 
flow  of  products  and  services.  The  SCM  process  be¬ 
gins  in  the  earliest  stages  of  procuring  materiel  from 
distributors  and  suppliers,  and  it  continues  until  the 
final  delivery  to  the  customer.  The  complete  supply 
chain  encompasses  sourcing  raw  materiel  and  parts, 
manufacturing,  warehousing,  inventory  management, 
customer  service,  and  delivery  of  final  product.  Each 
element  is  critical  for  a  fully  developed,  high-perform¬ 
ing  supply  chain;  one  weak  link  can  prove  detrimental 
to  the  entire  SCM  process  because  all  areas  depend 
upon  each  other.  SCM  aims  to  collectively  improve 
each  stage  of  product  acquisition,  storage,  develop¬ 
ment  and  delivery  to  ultimately  maximize  customer 
value  and  gain  a  more  competitive  advantage. 


An  established  supply  chain  optimizes  a  business'  opera¬ 
tions  by  enhancing  both  speed  and  efficiency  through¬ 
out  the  various  stages,  from  obtaining  raw  materiel  to 
delivering  the  final  product.  The  collaborative  efforts  of 
the  areas  within  the  supply  chain  give  companies  a  huge 
cost  advantage  due  to  the  greater  efficiency  within  each 
area  of  the  organization.  Advantages  to  implementing  an 
effective  supply  chain  include  improved  inventory  man¬ 
agement,  better  materiel  distribution,  higher  visibility  of 
assets,  increased  customer  satisfaction,  reduced  waste, 
lower  costs,  predictable  schedules,  enhanced  quality  and 
better  overall  operational  efficiency. 


O'Neall  is  a  Naval  Surface  Warfare  Center-Crane  Division  contract 
specialist  in  Crane ,  Indiana.  She  holds  a  bachelor's  degree  in  man¬ 
agement  from  Indiana  University-Purdue  University  Indianapolis  and 
an  MBA  from  Indiana  Wesleyan  University.  Haraburda,  a  retired 
U.S.  Army  colonel ,  is  the  Crane  Army  Ammunition  Activity  Strategic 
Planner.  He  holds  a  doctorate  degree  in  chemical  engineering  from 
Michigan  State  University  and  is  DAW  I A  Level  III  certified  in  program 
management  and  engineering.  He  also  is  a  certified  Project  Manage¬ 
ment  Professional. 
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Figure  1.  Phases  of  Military  Supply  Chains 
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SCM  Implementation 

An  effective  supply  chain  has  a  huge  impact  on  an  orga¬ 
nization's  profitability  and  customer  satisfaction,  and  this 
leads  to  a  greater  need  for  SCM  process  improvement.  SCM 
extends  beyond  traditional  logistics  to  include  suppliers  and 
customers  in  the  process,  resulting  in  more  involvement 
from  outside  stakeholders  and  better  overall  relationships. 
CAAA  has  begun  initiatives  to  integrate  a  fully  functioning 
supply  chain  system  with  the  more  engaging  approaches 
offered  by  SCM. 

CAAA  uses  the  Logistics  Modernization  Program  (LMP),  an 
Enterprise  Resource  Planning  (ERP)  software  program,  to 
manage  the  Army's  materiel,  monitor  and  increase  inventory 
accuracy,  reduce  cycle  time  of  purchase  orders,  and  improve 
overall  business  functions.  By  using  this  system,  CAAA's  sup¬ 
ply  chain  team  conducted  extensive  research  and  collected 
vast  amounts  of  data  to  move  SCM  integration  forward. 

New  innovative  supply  chain  processes  were  cultivated  by 
looking  outside  the  U.S.  Government  to  industry  leaders.  In  the 
earliest  stages  of  SCM  development,  CAAA  used  the  Ameri¬ 
can  Productivity  and  Quality  Center  (APQC)  for  information 
about  best  practices,  benchmarking,  performance  improve¬ 
ment  and  other  areas.  CAAA  also  used  the  American  Produc¬ 
tion  and  Inventory  Control  Society  (APICS)  for  information 
regarding  its  supply  chain  operations  reference  (SCOR)  model, 
which  linked  business  processes,  performance  metrics,  best 
practices  and  individual  skills  into  one  structure. 

Used  throughout  the  Department  of  Defense  (DoD),  the  SCOR 
framework  is  organized  around  six  primary  processes  of  plan, 
source,  make,  deliver,  return,  and  enable.  Because  this  model 
is  used  to  improve  and  communicate  supply  chain  goals  and 
processes,  it  assisted  CAAA  with  metric  identification  and 
arrangement.  It  also  has  been  further  developed  to  apply  to 
a  DoD  supply  chain.  As  seen  in  Figure  1,  for  military  supply 
chains,  each  phase  targets  specific  areas— and  they  repeat 


again  and  again  throughout  the  entire  process.  CAAA  selected 
metrics  reflected  in  this  model  that  focus  on  a  unified  SCM 
process  from  supplier  delivery  to  customer  acceptance. 

CAAA  initiated  a  key  initiative  to  establish  effective  perfor¬ 
mance-based  metrics  and  created  a  balanced  scorecard  to 
measure  supply  chain  efficiency.  It  used  this  balanced  score- 
card  as  a  performance  measurement  tool  to  provide  leaders  a 
complete  view  of  organizational  performance  across  various 
mission  and  support  areas.  This  was  done  primarily  through 
modifying  the  project  management  dashboards  originally 
developed  by  one  of  the  authors,  described  in  "Leveraging 
Fidelity  of  Performance-Based  Metric  Tools  for  Project  Man¬ 
agement,"  an  article  in  the  January-February  2003  issue  of 
Program  Manager ,  the  predecessor  of  Defense  AT&L  magazine. 

Identifying  Supply  Chain  Metrics 

Tracking  supply  chain  metrics  is  always  a  strong  starting  point 
because  the  metrics  allow  organizations  to  identify  weak  areas 
and  opportunities  for  performance  improvement.  Assisted 
by  APQC  information,  CAAA  identified  an  initial  list  of  nearly 
50  possible  metrics  to  use  in  determining  where  an  activity 
stands  in  terms  of  its  SCM  capability.  Table  1  lists  some  of 
those  metrics  that  focused  on  key  performance  indicators  cov¬ 
ering  process  efficiency,  cycle  time,  staff  productivity,  cost  ef¬ 
fectiveness  and  other  supplemental  information.  CAAA  used 
the  top  quartile  of  industrial  companies  as  benchmarked  target 
values  in  its  scorecard. 

During  the  initial  review,  CAAA  collected  the  appropriate  data 
from  the  LMP  database  to  analyze  the  activity's  strength  in 
the  above  key  performance  areas.  CAAA  leaders  wanted  to 
eliminate  "Zombie  Metrics"  that  were  causing  no  action,  even 
if  they  were  easier  to  monitor.  After  completion  of  this  initial 
assessment,  the  list  was  compressed  to  fewer  than  two  dozen 
metrics  to  target  the  weakest  areas,  such  as  inventory  man¬ 
agement,  supplier  performance  and  schedule.  Furthermore, 
the  weaker  links  in  the  supply  chain  were  paired  with  some  of 
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CAAA's  business  critical  metrics  such  as  productivity,  qual¬ 
ity,  customer  satisfaction,  and  revenue  and  expense  planning. 
The  final  metrics  were  arranged  into  the  elements  of  supplier, 
input,  process,  output,  customer,  and  foundation  (SIPOCF). 

Used  in  developing  the  balanced  scorecard  for  the  complete 
supply  chain,  CAAA's  supply  chain  metrics  were  those  com¬ 
posed  of  these  SIPOCF  elements,  and  used  these  in  various 
process  improvement  projects.  CAAA  pushed  the  SIPOCF 
diagram  a  step  further  by  adding  a  foundation  category  to  help 
view  the  entire  organization's  base  performance.  Each  group 
within  the  SIPOCF  diagram  consists  of  key  metrics  that  mea¬ 
sure  specific  areas  within  the  supply  chain  and  contribute  to 
completion  of  the  overall  balanced  scorecard. 

The  supply  element  consists  of  the  significant  internal  and  ex¬ 
ternal  suppliers  to  the  process.  In  Fiscal  Year  2015,  CAAA  had 
nearly  80  different  vendors,  both  commercial  and  government, 
supplying  more  than  100  distinct  components.  The  metrics 
used  in  this  stage  include  Materiel  Requirements  Planning  and 
product  quality  deficiency  data.  These  supplier  performance 
metrics  are  tied  to  key  enterprise  performance  metrics  and 
processes  rather  than  a  sole  emphasis  on  establishing  supplier 
performance  targets  without  organizational  considerations. 

The  input  element  consists  of  the  materiel,  resources,  docu¬ 
ments,  information  or  any  other  significant  data  used  to  imple¬ 
ment  the  organizational  processes.  CAAA  measures  its  signifi¬ 
cant  inputs  by  analyzing  inventory  turns,  days  of  supply,  supply 
planning,  and  demand  and  supply  forecasting.  Regrettably, 
many  DoD  installations— CAAA  included— have  increasing 
inventory  levels,  which  complicates  the  supply  chain  process 


Table  1.  Possible  Supply  Chain  Metrics 


Category 

Metric 

Process  Efficiency 

Raw  materiel  inventory  turns. 

Cycle  Time 

Time  to  place  a  purchase  order. 

Supplemental  Info 

Percentage  of  procurement  group  with 
more  than  3  years  of  experience. 

Staff  Productivity 

Number  of  purchase  orders  processed 
per  procurement  group  full  time  equiva¬ 
lent. 

Cost  Effectiveness 

Total  cost  to  process  outbound  transpor¬ 
tation  per  $1,000  in  revenue. 

Source:  American  Productivity  and  Quality  Center 


and  damages  overall  performance.  Measuring  and  improving 
inventory  management  enhances  supply  and  demand  planning 
and  the  business'  overall  profitability. 

The  process  element  consists  of  the  activities  or  procedures 
executed  in  an  effort  to  convert  inputs  into  outputs.  The  ef¬ 
ficiency  processes  are  measured  through  productive  yield 
calculations,  receipts  and  issues,  inventory  plan  completion 
and  scrap  and  rework  costs.  Process  metrics  allow  CAAA 
leaders  to  examine  operations  and  productivity  levels  across 
multiple  functions. 

The  output  element  consists  of  items  produced  in  the  pro¬ 
cess  phase  and  intended  for  customer  use.  Metrics  include 
the  effectiveness  of  their  outputs  through  the  quality  of  end 
items,  the  number  of  projects  completed,  the  schedule,  orders 


Figure  2.  Actionable  Data  From  Supply  Chain  Metrics 
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shipped  on  time,  surveillance  backlog  and  demilitarization 
execution.  These  provided  leaders  with  an  idea  of  the  qual¬ 
ity  and  timeliness  of  the  various  products  generated  by  each 
project.  Monitoring  outputs  also  increases  the  visibility  of 
process  outcomes  and  provides  an  opportunity  to  continu¬ 
ously  modify  operating  procedures  to  improve  the  quality 
and  schedule  of  end  products. 

The  customer  element  involves  the  recipient  of  the  output 
from  the  process.  CAAA  has  slightly  more  than  two  dozen 
different  customers  who  receive  its  products.  Metrics  are 
applied  to  quality,  timeliness  and  delivery.  These  impact 
future  demand,  ensure  customer  satisfaction,  and  identify 
opportunities  to  improve.  These  included  customer  feed¬ 
back,  especially  their  complaints.  Perfect  Order  Fulfillment 
also  is  tracked  to  ensure  products  are  supplied  to  the  cus¬ 
tomer  on  schedule  and  meet  all  requirements  agreed  upon 
in  the  contract. 

The  foundation  element  refers  to  the  enabling  processes 
in  the  SCOR  model.  These  metrics  included  such  enabling 
processes  as  safety,  direct  labor  hours,  revenue  and  expense 
planning— along  with  storage  occupancy  levels. 

Balanced  Scorecard 

Action-Based  Reviews:  At  least  once  each  month,  the  CAAA 
directors  review  actionable  information  determined  from  an 
effective  analysis  of  their  supply  chain  metrics.  The  monthly 
metrics  reviews  include  quad-charts  similar  to  the  chart  in 
Figure  2,  one  for  each  of  the  six  SIPOCF  elements.  These 
reviews  include  Green-Amber-Red  (GAR)  colors  along  with 
trending  arrows  to  indicate  changes  from  previous  reports 
(Is sK-».  The  GAR  colors  are 
based  upon  the  target  goals, 
either  established  by  the 
command  or  from  the  perfor¬ 
mance  of  companies  through 
benchmarking  target  data. 


Dashboard:  CAAA  integrates  the  six  SIPOCF  elements  into 
an  integrated  dashboard,  similar  to  the  balanced  scorecard 
used  by  the  Malcolm  Baldrige  National  Quality  Award.  This 
shows  its  leaders  the  status  of  its  organization  in  a  quick 
glance.  Cost,  schedule  and  performance  are  the  overarching 
three  key  business  indicators  that,  as  in  project  management, 
are  displayed  on  the  top  of  this  dashboard.  The  foundation 
element  is  displayed  on  the  bottom  and  the  remaining  five 
elements  are  displayed  in  the  middle  in  sequential  order. 
As  in  the  action-based  reviews,  each  of  the  elemental  bar 
charts  includes  a  trending  arrow.  Resembling  a  hamburger, 
this  dashboard  looks  like  the  notional  one  in  Figure  3. 

The  Balanced  Scorecard’s  Future 

The  balanced  scorecard  based  upon  SCM  has  developed  into 
an  integral  part  of  CAAA's  strategic  and  operational  plans 
used  by  its  leaders  to  make  important  organizational  deci¬ 
sions  throughout  the  month.  It  also  provides  a  range  of  per¬ 
formance  metrics  that  CAAA  leaders  use  in  making  informed 
decisions  involving  supply  chain  design  changes.  Any  organi¬ 
zation,  including  service-oriented  government  agencies,  can 
benefit  by  using  a  balanced  scorecard  of  key  performance 
metrics  with  industrial  benchmarks  in  assessing  organiza¬ 
tional  progress  and  identifying  weaker  links  throughout  its 
supply  chains.  & 

The  authors  can  be  contacted  at  casandra.e.oneall@navy.mil  and 
scott.s.haraburda.civ@mail.mil. 


Figure  3.  Supply  Chain  Dashboard  (With  Trends) 

Business 
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elemental  metrics  in  each  of 
the  GAR  rating.  The  upper 
right  block  contains  the  data 
for  each  of  the  elemental 
metrics,  including  the  value 
from  the  previous  report¬ 
ing  period.  The  bottom  left 
block  provides  a  textual  list 
of  the  top  issues  in  the  supply 
chain  element.  And  the  bot¬ 
tom  right  block,  as  the  most 
important  one  in  the  review, 
identifies  the  action  plan  to 
address  the  identified  issues. 
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Diminishing  manufacturing  sources  and 
material  shortages  (DMSMS)  remain 
a  very  significant  issue  for  the  Depart¬ 
ment  of  Defense  (DoD),  with  signifi¬ 
cant  resources  committed  to  limiting 
the  problem.  Given  the  long  lives  of  DoD  systems 
relative  to  the  items  and  technologies  used  to 
build  and  support  them,  DMSMS  problems  are 
inevitable.  There  is  good  news,  however:  Proac¬ 
tive  DMSMS  management  can  reduce  the  cost  of 
resolving  those  problems. 

How  does  proactive  DMSMS  management  help?  It's  all  about  the 
window  of  opportunity  to  do  something  about  emerging  DMSMS 
issues.  Proactive  DMSMS  management  reduces  cost  by  identifying 
issues  as  early  as  possible  through  a  risk-based  monitoring  of  items 
in  the  system.  If  a  program  does  not  discover  a  DMSMS  issue  until 
there  is  a  failed  attempt  to  buy  an  item,  resolution  options  often  are 
limited  and  usually  only  more  expensive  alternatives  are  feasible. 
Proactively  identifying  issues  as  soon  as  information  about  them 
becomes  available  usually  increases  the  number  of  resolution  op¬ 
tions  available  and  creates  opportunities  for  an  increased  number  of 
lower-cost  alternatives  because  there  is  more  time  to  fix  the  problem 
before  an  impact  occurs. 

This  article  illustrates  DMSMS  management  efficiencies  achieved  via 
collaboration  between  the  Naval  Air  Systems  Command  (NAVAIR) 
and  the  U.S.  Army  Aviation  and  Missile  Research,  Development,  and 
Engineering  Center  (AMRDEC).  These  efficiencies  lead  to  greater 
team  proactivity  and  ultimately  to  lower  DMSMS  resolution  costs 
for  the  partnering  organizations. 


Mandelbaum  was  instrumental  in  developing  ways  to  use  value  engineering  to 
resolve  obsolescence  issues  in  the  course  of  researching  obsolescence  policy,  guid¬ 
ance,  and  training  during  the  last  7 years  at  the  Institute  for  Defense  Analysis  (IDA). 
Patterson  also  has  worked  for  7  years  on  obsolescence  policy  at  IDA  and  earlier  was 
similarly  involved  in  the  systems  engineering  field.  Radford  has  more  than  15  years 
of  experience  in  Engineering,  Logistics  and  Program  Management  within  industry 
and  the  government,  supporting  aviation  and  ground  support  platforms.  Alcorn 
has  worked  for  more  than  26  years  with  the  Army's  Aviation  and  Missile  Research, 
Development  and  Engineering  Center ;  and  is  currently  Team  Lead  and  Acting  Chief 
of  the  Obsolescence  Engineering  Branch.  Conroy  has  been  a  professor  of  Life  Cycle 
Logistics  Management  and  of  Production,  Quality  and  Engineering  at  the  Defense 
Acquisition  University's  Mid-Atlantic  campus  in  California,  Maryland,  since  2005. 
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The  DoD  guidebook,  SD-22,  Diminishing  Manufactur¬ 
ing  Sources  and  Material  Shortages:  A  Guidebook  of  Best 
Practices  for  Implementing  a  Robust  DMSMS  Manage¬ 
ment  Program ,  (http://www.acq.osd.mil/se/docs/SD- 
22-DMSMS.pdf),  provides  comprehensive  information 
for  DoD  Components  on  mitigating  the  risks  associated 
with  DMSMS  issues.  It  defines  DMSMS  as  "the  loss,  or 
impending  loss,  of  manufacturers  or  suppliers  of  items, 
raw  materials,  or  software."  The  SD-22  also  defines 
DMSMS  management  as  "a  multidisciplinary  process 
to  identify  issues  resulting  from  obsolescence,  loss  of 
manufacturing  sources,  or  material  shortages;  to  assess 
the  potential  for  negative  impacts  on  schedule  and/or 
readiness;  to  analyze  potential  mitigation  strategies;  and 
then  to  implement  the  most  cost-effective  strategy." 


Cross-Service  Collaboration 

NAVAIR's  DMSMS  management  team  is  the  logistics  techni¬ 
cal  authority  responsible  for  the  development,  sustainment 
and  execution  of  NAVAIR  DMSMS  and  obsolescence  man¬ 
agement  policy  and  processes.  Its  mission  is  to  mitigate  the 
impact  of  obsolescence  and  DMSMS  issues  on  total  ownership 
cost  by  providing  relevant  subject-matter  expertise  to  N  AVAI R 
program  management  offices.  The  AMRDEC  Obsolescence 
Engineering  team  includes  more  than  40  engineers  and  re¬ 
searchers  who  develop  obsolescence  engineering  strategies 
and  constantly  monitor  products  and  parts  availability  issues. 
Since  1987,  the  team,  located  within  AMRDEC's  Engineering 
Directorate,  has  supported  aviation  and  missile  programs 
combating  the  ever-present  threat  of  obsolescence. 

Both  collaborating  organizations  previously  performed  similar 
functions  independently— they  both  apply  tools  and  resources 
as  part  of  their  support  to  programs.  For  example,  the  teams' 
research  functions: 

■  Facilitate  the  analysis  of  Bills  of  Materials  (BOMs)  using 
a  suite  of  predictive  tools  primarily  for  monitoring  elec¬ 
tronic  parts. 

■  Perform  market  research  by  contacting  the  applicable 
vendors  to  ensure  that  accurate  data  (e.g.,  points  of 
contact,  pricing,  end  of  production,  etc.)  are  obtained  for 
mechanical  and  commercial-off-the-shelf  (COTS)  parts. 

■  Access  the  original  equipment  manufacturer  and  after- 
market  manufacturer  websites. 

■  Access  federal  supply  sources  such  as  Defense  Logistics 
Agency  (DLA),  Land  and  Maritime,  Standard  Microcircuit 
Cross  Reference,  Qualified  Products  Database,  as  well  as 
additional  commercial  tools. 

The  need  for  a  centralized  database  and/or  tool  to  facilitate 
these  research  functions  and  disseminate  the  results  became 
evident  as  the  number  of  programs  supported  by  the  NAVAIR 
DMSMS  team  grew.  After  evaluating  a  range  of  potential  op¬ 
tions,  NAVAIR  selected  the  Multifunctional  Obsolescence 
Resolution  Environment  (MORE)  tool-database  to  meet  its 


needs.  MORE  is  a  government-owned  engineering  analytical 
obsolescence  and  DMSMS  management  information  system 
developed  and  maintained  by  AMRDEC.  MORE  centralizes 
work  flow  for  researching  the  status  of  electronic  parts;  ac¬ 
cessing  availability  data,  analysis  results,  and  discontinuance 
alerts;  and  compiling  and  disseminating  of  information  gleaned 
from  subject-matter  experts. 

While  the  possession  of  these  capabilities  influenced  the 
selection  of  MORE,  an  even  greater  factor  in  favor  of  MORE 
was  AMRDEC's  willingness  to  truly  partner  with  the  NAVAIR 
team.  Thus,  cross-Service  DMSMS  management  collaboration 
was  born!  The  following  illustrates  examples  of  the  synergies 
that  have  already  been  gained  via  this  real-world  joint,  multi- 
Service  DMSMS/obsolescence  partnership. 

Benefits  of  the  Collaboration 

During  the  first  year  of  partnering,  the  MORE  parts  library 
increased  in  size  by  approximately  one-third  when  NAVAIR 
parts  were  included.  Further  increases  are  anticipated  be¬ 
cause  only  a  small  portion  of  the  total  NAVAIR  BOMs  were 
loaded  initially.  Just  from  the  perspective  of  NAVAIR  alone, 
when  these  initial  BOMs  were  loaded,  more  than  15  percent 
of  the  parts  were  already  common  to  the  AMRDEC  and  had 
been  researched  and  were  in  the  MORE  library.  This  15  percent 
likely  will  increase  because  many  of  the  NAVAIR  parts  must 
be  researched  prior  to  a  determination  of  whether  they  are  in 
the  parts  library. 

With  an  expanded  parts  library,  there  is  a  greater  likelihood 
that  any  parts  investigated  by  new  AMRDEC  or  NAVAIR  cus¬ 
tomers  not  only  will  already  be  in  the  library  but  can  also  be 
automatically  researched.  And  this  provides  a  significant  time 
savings  over  a  manual  process.  In  fact,  beyond  the  15  percent 
commonality,  many  of  the  initial  NAVAIR  unique  parts  were 
automatically  researched  by  MORE.  Finally,  the  increase  in 
the  MORE  parts  library  leads  to  an  increased  number  of  parts 
added  to  subscription  tools  that  underlie  MORE.  Over  time, 
this  will  increase  the  recognition  rate  of  the  parts  libraries 
within  the  subscription  tools. 

NAVAIR  Obsolescence  Management  Team  Technical  Lead 
Chris  Radford  says:  "The  MORE  library  when  combined  with 
the  AMRDEC  team  provides  a  service  and  capability  that  no 
one  else  in  the  industry  provides.  It's  a  one-stop  shop  with 
a  program  designed  around  MORE  that  provides  not  only 
complete  documented  work  instructions  from  part  research 
to  program  management  of  DMSMS,  but  also  a  unique  part 
auditing  program  that  ensures  that  bad  data  and  research  are 
not  added  to  the  library,  either  willingly  or  unwillingly." 

Radford  added:  "This  is  the  key  to  a  successful  DMSMS 
program.  No  matter  what  tools  you  use,  there  is  a  high 
percentage  of  false  data  that  exists.  The  MORE  process  is 
constantly  reducing  this  bad  data  to  ensure  that  its  contents 
are  more  accurate  than  any  other  tool  because  the  data  is 
validated.  This  proactive  management  process  is  the  key  to 
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DMSMS  cost  avoidance  based  on  program  efficiency.  Most 
programs  are  still  searching  for  the  individual  part  metrics 
hoping  to  save  big  dollars  not  realizing  that  the  manpower 
costs  they  are  spending  to  reactively  solve  these  problems 
generally  offset  the  costs  saved.  Good  DMSMS  programs 
don't  save  their  customers  money  overnight— rather,  they 
establish  a  consistent  program  that  enables  the  customers 
to  proactively  manage  all  of  their  parts  and  systems  and 
streamline  future  efforts.” 

Both  NAVAIR's  and  AMRDEC's  proficiency  have  increased 
as  a  result  of  the  collaboration.  Although  NAVAIR  also 
uses  other  tools  to  facilitate  the  research  and  identification 
of  alternate  parts  and  part  statuses,  MORE  leverages  the 
information  provided  by  those  tools.  It  enhances  and  com¬ 
pares  their  outputs,  thereby  providing  the  user  with  more 
accurate  parts  availability  statuses. 

The  AV-8B  ground-attack  aircraft  program  offers  several  ex¬ 
amples  of  efficiencies  already  realized  because  of  the  partner¬ 
ship.  The  AV-8B  leveraged  AMRDEC  DMSMS  training  docu¬ 
mentation  not  only  to  train  new  staff  on  how  to  use  MORE 
but  also  on  how  the  parts  research  process  works.  The  MORE 


Instruction  and  is  collaborating  on  requirements  for  a  MORE 
Sustainment  Module  currently  in  development. 

Michele  Ozier,  a  team  lead  at  AMRDEC,  and,  in  particular, 
the  AMRDEC  lead  for  the  NAVAIR  collaboration,  spoke  of  the 
mutual  benefits  of  the  combined  efforts:  "We  at  AMRDEC 
are  excited  to  collaborate  with  NAVAIR.  We  believe  that  the 
resulting  identification  of  commonality,  standardization  of 
processes,  shared  ideas,  and  synergy  will  be  a  great  benefit 
to  both  organizations,  and  most  importantly,  to  our  custom¬ 
ers— the  warfighters.” 

MORE  can  also  facilitate  determining  resolution  options 
to  DMSMS  problems  through  a  capability  to  view  all  plat¬ 
forms  that  are  using  a  given  part.  Consequently,  when  a 
program  office  is  trying  to  determine  the  most  cost-effec¬ 
tive  resolution  to  a  common  problem,  it  can  easily  identify 
what  other  platforms  have  done  and  take  advantage  of 
those  efforts.  In  addition,  the  ability  to  view  all  platforms 
that  use  a  part  can  better  enable  a  coordinated  cross- 
Services  programs  approach  to  parts  suppliers  for  resolu¬ 
tions,  thereby  increasing  their  leverage  to  either  get  the 
suppliers  to  continue  manufacturing  the  item  or  provide 


Good  DMSMS  programs  don't  save  their  customers  money 
overnight— rather,  they  establish  a  consistent  program  that  enables 
the  customers  to  proactively  manage  all  of  their  parts  and  systems 

and  streamline  future  efforts. 


partnership  allowed  the  AV-8B  team  to  streamline  its  obso¬ 
lescence  team,  process  and  structure;  it  is  now  managed  by  a 
small  core  group,  reducing  costs  from  $633,000  in  Fiscal  Year 
(FY)  2014  to  $290,000  for  FY  2017.  Jesse  Powell,  the  AV-8B 
Obsolescence  Manager,  said:  "Collaboration  and  leveraging 
existing  processes  and  tools  across  the  Services  should  be  our 
Number  One  goal.  We  often  spend  too  much  time  doing  our 
job  that  we  forget  that  there  are  other  Services  within  DoD  that 
may  have  already  solved  the  problem.  The  Army  and  NAVAIR 
collaboration  through  the  MORE  tool  is  just  one  example  of 
how  we  (all  DoD)  can  work  together  to  reach  a  common  goal. 
The  AV-8B  program  at  NAVAIR  has  shown  that  collaboration 
can  lead  to  saving  for  the  program  office,  and  I  look  forward 
to  continuing  this  collaboration  into  the  future.” 

AMRDEC's  part  research  proficiency  has  also  improved  as  a 
result  of  the  NAVAIR  addition  of  a  large  number  of  Military 
Specification  (MILSPEC)  items  into  the  MORE  database.  In  ad¬ 
dition,  NAVAIR  input  on  AMRDEC  processes  helped  to  further 
refine  and  enhance  MORE  processes.  Specifically,  NAVAIR 
reviewed  and  provided  comments  on  MORE's  MILSPEC  Work 


an  alternative.  DoD's  bargaining  power  can  be  further  en¬ 
hanced  because  a  single  supplier  often  provides  multiple 
items  to  multiple  DoD  customers. 

Taking  Collaboration  to  the  Next  Level 

The  AMRDEC/NAVAIR  collaboration  in  the  use  of  the 
MORE  tool  represents  just  one  of  eight  strategic  objec¬ 
tives  being  pursued  to  expand  collaboration  across  the 
whole  DoD  enterprise.  Another  of  those  objectives  is  com¬ 
monality.  The  goal  of  this  strategic  objective  is  to  dem¬ 
onstrate  the  value  (including  reduced  costs,  improved 
program  schedule  and  other  efficiencies)  of  a  proactive 
DMSMS  program  leveraging  information  sharing.  This  ob¬ 
jective  was  created  in  recognition  of  lost  opportunities  for 
common  resolutions.  Data  sharing  previously  occurred 
only  within  a  Service  among  the  customers  of  the  same 
independent  DMSMS  management  provider,  the  users  of 
common  tools,  or  as  a  result  of  periodic  meetings  of 
various  working  groups.  Sharing  also  occurred  across 
DoD  where  common  resolutions  were  developed  for 
DLA-managed  electronic  items— or  in  rare  instances, 
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such  as  tungsten-rhenium  wire,  when  it  was  deter¬ 
mined  that  an  enterprise  resolution  was  preferred. 

A  third  strategic  objective  deals  with  the  establishment  of 
DoD  centers  of  excellence.  When  a  DMSMS  problem  occurs, 
resolution  options  are  analyzed  to  determine  the  most  cost- 
effective  approach.  The  comprehensiveness  of  the  analysis 
depends  primarily  on  a  program  office's  experiences  with  the 
capabilities  of  resolution  providers.  This  experience  is  typi¬ 
cally  limited  because  program  offices  often  choose  only  from 
the  subset  of  potential  service  providers  that  they  commonly 
work  with.  The  goal  of  this  strategic  objective  is  to  create  an 
easy-to-use  database  of  a  large  number  of  service  provider 
capabilities  that  program  offices  can  use  to  help  determine 
the  most  cost-effective  approach  for  resolving  DMSMS  issues. 

These  strategic  objectives  are  only  achievable  through  various 
forms  of  partnerships.  The  NAVAIR  and  AMRDEC  partner¬ 
ship  is  not  just  one  example  but  a  first  step.  Robin  Brown,  the 
DoD  DMSMS/Obsolescence  lead  said,  "I  want  to  make  the 
case  for  us  to  build  on  this  partnership  to  include  not  only  all 
DMSMS  management  across  DoD  but  also  the  resources  that 
DMSMS  practitioners  rely  upon  to  resolve  problems.”  Benefits 
are  already  being  witnessed  through  the  NAVAIR/AMRDEC 


partnership;  therefore,  it  is  credible  that  further  benefits  can 
be  realized  by  expanding  the  collaboration  further  across  the 
DoD  enterprise. 

Conclusion 

While  the  Services  use  numerous  unique  systems  and  plat¬ 
forms,  it  is  important  to  understand  that  many  common 
components  exist  on  these  systems,  regardless  of  func¬ 
tion,  application  or  the  environments  in  which  they  per¬ 
form.  In  the  past,  because  of  how  programs  are  segregated 
and  managed,  these  common  parts  were  likely  monitored 
and  researched  independently  by  multiple  programs  or  not 
tracked  at  all.  This  has  resulted  in  duplicated  effort  and 
inefficient  use  of  resources.  Collaboration,  enabled  by  a 
centralized  database,  delivers  benefits  to  all  players  in¬ 
volved  in  component  research  and  mitigation  by  reducing 
time  and  cost  and  by  utilizing  a  team  of  subject-matter  ex¬ 
perts  (rather  than  a  single  person)  to  participate  in  reducing 
DMSMS  risks.  These  efficiencies  lead  to  improved  proac¬ 
tive  DMSMS  management  and  thereby  decreased  DMSMS 
management  and  resolution  costs.  & 

The  authors  can  be  contacted  atjmandelb@ida.org;  cpattersaida. 
org;  christopher.radford@navy.mil;  allen.s.alcorn.civ@mail.mil;  and 
william.conroy@dau.mil. 
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Got  opinions  to  air? 

Interested  in  passing  on  lessons  learned  from  your 
project  or  program? 

Willing  to  share  your  expertise  with  the 
acquisition  community? 

Want  to  help  change  the  way  DoD  does  business? 

Write  an  article  (1,500  to  2,500  words)  and  Defense  AT&L 
will  consider  it  for  publication.  Our  readers  are  interested  in 
real-life,  hands-on  experiences  that  will  help  them  expand  their 
knowledge  and  do  their  jobs  better. 


What’s  in  It  for  You? 

First  off,  seeing  your  name  in  print  is  quite  a  kick.  But  more 
than  that,  publishing  in  Defense  AT&L  can  help  advance  your 
career.  One  of  our  authors  has  even  been  offered  jobs  on  the 
basis  of  articles  written  for  the  magazine. 

Now  we  can’t  promise  you  a  new  job,  but  many  of  our  authors: 

•  Earn  continuous  learning  points 

•  Gain  recognition  as  subject-matter  experts 

•  Are  invited  to  speak  at  conferences  or  symposia 

•  Get  promoted  or  rewarded 


For  more  information  and  advice  on  how  to  submit  your  manuscript,  check  the  writer’s  guidelines  at 
https://www.dau.miI/library/defense-atl/p/Writers-Guidelines  or  contact  the  managing  editor  at  datl@dau.mil. 
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If  Only  Our  Training  Could 


Brian  Schultz  ■  Jeffrey  W.  Megargel 


Could  our  acquisition  training  be  focused  on  our 
program  and  involve  our  unique  issues?  Could 
the  training  be  comprehensive  enough  to  help 
our  team  with  a  critical  task  such  as  developing 
an  Acquisition  Strategy?  Could  program-specific 
Mission  Assistance  be  provided  when  our  team  first  needs 
it,  like  at  the  beginning  of  the  acquisition  life  cycle  or  before 


"Tell  me 
and  I  forget. 
Teach  me  and 
I  remember. 
Involve  me 
and  I  learn." 

—Adage,  Various 
Attributions 


Schultz  is  a  professor  in  the  Defense  Acquisition  University's  Capital  and  Northeast  Region  at  Fort 
Belvoir,  Virginia.  Megargel  is  a  former  Marine  Corps  Contracting  Officer  and  industry  program  man¬ 
ager  currently  supporting  the  Joint  Program  Executive  Office  for  Chemical  and  Biological  Defense 
Contracting  Management  Office. 
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a  major  milestone?  Could  the  training 
come  to  us  so  we  can  ensure  we  have  the 
right  staff  available?  Could  the  training 
employ  team-based  critical  thinkingtech- 
niques  that  we  can  use  after  the  training 
event?  Could  it  even  be  fun? 

Believe  it  or  not,  the  answer  to  all  these 
questions  is  yes! 

The  Joint  Program  Executive  Office  for 
Chemical  and  Biological  Defense  (JPEO- 
CBD)  at  Aberdeen  Proving  Ground,  Edge- 
wood  Area,  in  Maryland,  and  the  Defense 
Acquisition  University  (DAU)  have  part¬ 
nered  to  develop  a  new,  1-week  Mission 
Assistance  workshop  that  does  all  these 
things  and  more. 


An  added  benefit 
of  conducting  this 
workshop  early  in  the 
program  life  cycle  is 
the  identification  of 
specific  help  that  the 
team  may  need. 


The  new  workshop,  Acquisition  Strategy 
Development  Workshop  (WSM  014  in 
DAU's  i-catalog),  is  designed  to  provide 
program  teams  with  training  that  helps 
streamline  their  Acquisition  Strategy  in  preparation  for  the 
Milestone  Decision  Authority  review.  The  training  challenges 
the  teams  to  tailor  their  acquisition  strategies  with  a  focus  on 
eliminating  non-value-added  effort  and  delivering  high-value 
capability  within  programmatic  constraints.  It  also  attempts 
to  enable  more  informed  decision  making  early  in  the  pro¬ 
gram  life  cycle.  While  the  JPEO-CBD  has  primarily  targeted 
Acquisition  Category  (ACAT)  III  programs  for  the  workshop, 
it  can  easily  be  tailored  to  ACAT  I  and  II  programs.  ACATs 
facilitate  appropriate  decision  making  and  oversight  of  acqui¬ 
sition  programs  based  on  funding  and  interest  levels.  ACAT  I 
programs  are  the  most  costly  and  have  the  greatest  reporting 
and  documentation  requirements.  The  budgetary  amounts 
designated  decline  for  ACAT  I  Is  and  ACAT  Ills,  as  well  as  the 
reporting  and  documentation  requirements. 


Because  an  Acquisition  Strategy  in¬ 
volves  technical,  business  and  support- 
ability  elements,  the  workshop  content 
to  support  this  task  needed  to  be  broad 
but  relevant.  The  workshop  agenda 
typically  includes  more  than  14  different 
topics  that  cut  across  functional  areas. 
Understanding  the  policy  and  proven 
practices  is  a  good  starting  point,  but 
the  challenge  is  to  determine  how  best 
to  apply  the  tools  to  each  program.  As 
part  of  each  module  in  the  agenda,  a 
discussion  is  facilitated  of  how  the  team 
will  address  each  area  in  its  strategy.  In¬ 
tegration  is  required  on  other  past  topics  and  content,  since 
some  of  the  content  builds  on  earlier  assessments.  To  facilitate 
this  task,  the  team  members  are  provided  a  briefing  template 
to  fill  out  as  they  progress  through  the  workshop.  The  event 
culminates  with  a  debriefing  to  examine  the  team's  ideas  and 
assessments  while  a  final  strategy  is  developed. 

Finally,  a  third  driver  is  the  need  to  help  teams  avoid  com¬ 
mon  inception  issues  through  critical  thinking  techniques 
introduced  at  the  beginning  of  the  training  and  used 
throughout  the  week  to  reinforce  the  value  of  good  think¬ 
ing  in  designing  a  path  forward.  Root  cause  analyses  of 
programs  incurring  significant  cost  and  schedule  growth 
often  point  to  program  inception  issues  when  program  de¬ 
velopment  is  just  starting. 


to  participate  in  the  full  event.  That  full 
participation  is  needed  because  several 
team  exercises  during  the  event  require 
an  IPT  perspective.  For  example,  techni¬ 
cal,  financial  and  supportability  risks  are 
identified  as  parts  of  overall  risk  identifi¬ 
cation.  The  team  members  then  discuss 
how  they  plan  to  mitigate  the  risks. 


The  need  for  this  new  workshop  was  driven  by  a  few  key  fac¬ 
tors.  At  the  JPEO-CBD,  the  most  important  factor  was  reduc¬ 
ing  cycle  time.  The  JPEO  has  determined  that  ACAT  III  pro¬ 
gram  managers  can  find  efficiencies  by  identifying  and  acting 
on  opportunities  to  reduce  or  eliminate  burdens  associated 
with  larger  ACAT  I  programs.  The  key  is  to  obtain  better  value 
in  balancing  programmatic  constraints  with  risk.  For  example, 
a  small  ACAT  III  program  may  be  planning  for  significant  use 
of  commercial  off-the  shelf  (COTS)  and  nondevelopmental 
items.  This  planning  assumption  suggests  that  several  docu¬ 
ments  and  tasks  focused  on  technology  risk  would  not  be 
needed  or  could  be  tailored  to  reflect  the  nature  of  the  devel¬ 
opmental  risk. 

A  second  driver  for  the  workshop  was  the  need  for  a  com¬ 
prehensive  training  effort  that  pulls  together  the  multiple 
functional  areas  of  an  integrated  product  team  (IPT).  One  of 
the  workshop's  tenets  is  that  key  functional  team  leaders  are 


A  good  example  is  a  framing  assumption— a  fundamental 
belief  about  program  conditions  that  we  expect  to  remain 
true  in  the  future.  If  such  an  assumption  proves  to  be  wrong, 
then  the  program's  cost,  schedule  and  performance  expec¬ 
tations  could  also  be  invalid  and  create  significant  problems 
in  meeting  expectations.  For  example,  if  the  team  believes 
that  significant  use  of  COTS  equipment  will  help  satisfy  the 
requirement,  this  belief  then  drives  cost,  schedule  and  per¬ 
formance  expectations.  If  it  turns  out  that  very  little  COTS 
can  be  used,  then  cost  estimates  and  schedule  durations  will 
be  significantly  different. 

As  part  of  the  workshop,  the  team  develops  program  framing 
assumptions.  The  assumptions  then  are  examined  carefully  to 
ensure  they  meet  the  criteria  for  good  assumptions.  Finally,  the 
team  members  discuss  their  plan  to  periodically  review  the  as¬ 
sumptions  and  make  any  needed  adjustments.  Techniques  to 
avoid  other  inception  problems— such  as  poor  risk  management, 
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lack  of  robust  systems  engineering,  and  inadequate  planning  for 
supportability— are  also  addressed  during  the  workshop. 

DAU  and  JPEO-CBD  recently  conducted  the  first  offerings 
of  the  workshop  at  Aberdeen  Proving  Ground.  In  addition  to 
the  program  teams,  JPEO-CBD  headquarters  staff,  the  Army 
Program  Executive  Office  (PEO)  for  Command,  Control  and 
Communications-Tactical  staff,  and  the  PEO  staff  for  Intel¬ 
ligence,  Electronic  Warfare  and  Sensors  attended  in  order  to 
observe  and  evaluate  the  workshop.  Based  on  feedback  from 
participants  and  observers,  the  workshops  were  very  success¬ 
ful  and  helped  the  team  to  develop  an  excellent  strategy  and 
planningfoundation.  The  workshops  culminated  with  debrief¬ 
ings  about  the  team's  developed  content,  including  several 
streamlining  proposals  and  ideas  to  improve  program  execu¬ 
tion  efficiency.  Valuable  feedback  from  participants  enabled 
the  DAU  workshop  team  to  make  some  initial  modifications 
to  the  overall  flow  and  content. 

An  added  benefit  of  conducting  this  workshop  early  in  the 
program  life  cycle  is  the  identification  of  specific  help  that  the 
team  may  need.  Since  the  workshop  covers  a  very  broad  range 
of  topics,  spending  more  than  a  few  hours  on  each  usually  is 
impractical.  The  team  members  may  realize  that  they  possess 
little  experience  or  skills  in  a  particular  area.  During  discus¬ 
sions  and  next  steps,  additional  resources  (including  training 
events)  can  be  identified  to  help  fill  those  gaps  as  the  team 
moves  forward,  both  pre-  and  post-award. 

Indeed,  DAU  and  JPEO-CBD  have  developed  a  series  of  tai¬ 
lored  workshops  that  are  designed  to  train  acquisition  work¬ 
force  to  comply  with  and  excel  within  the  latest  regulatory 
and  policy  guidance.  These  workshops  are  available  to  any 
Army  PEO  within  the  Office  of  the  Assistant  Secretary  of  the 
Army  for  Acquisition,  Logistics  and  Technology  and  to  other 
acquisition  organizations.  The  workshops  include: 

■  Contracting  for  Program  Managers  which  provides  an  over¬ 
view  of  the  revised  Department  of  Defense  (DoD)  Instruc¬ 
tion  5000.02  and  contracting  strategy,  contract  funding, 
contracting  incentives,  contract  types,  updates  to  the  Fed¬ 
eral  Acquisition  Regulation  (FAR),  DoD  contracting  initia¬ 
tives  and  their  impact  on  programs. 

■  Other  Transaction  Authority  (OTA)  (Basic  and  Advanced), 
designed  to  orient  program  teams  on  OTAs  and  to  indicate 
when  they  are  appropriate. 

■  A  Contract  Incentives  Workshop  that  teaches  development 
and  implementation  of  contract  incentives. 

■  A  Source  Selection  Workshop  that  provides  prospective 
Source  Selection  Evaluation  Board  members  with  an  over¬ 
view  of  the  source  selection  process  for  competitively  ne¬ 
gotiated  procurements  under  FAR  Part  15. 

■  An  Acquisition  Program  Transition  Workshop  held  shortly 
after  a  contract  is  awarded.  This  workshop  is  conducted 
with  the  government  program  team  and  the  winning  offeror 
and  seeks  to  align  the  efforts  of  both  parties  in  managing 
risk  and  deliverables. 


■  Other  possible  training  includes  risk  management  work¬ 
shops  and  various  types  of  systems  engineering  training. 
During  the  workshop,  the  DAU  instructor  also  will  identify 
appropriate  DAU  online  tools  and  products  to  supplement 
the  participants'  resident  training. 

While  this  acquisition  training  now  is  part  of  the  JPEO-CBD 
rhythm  for  new  AC  AT  III  programs,  other  organizations  could 
benefit.  To  that  end,  other  Army  PEO  programs  have  been  in¬ 
vited  to  start  participating  in  these  workshops.  DAU  is  working 
with  the  JPEO-CBD,  the  Army  and  other  DoD  acquisition  stake¬ 
holders  to  assess  the  demand  for  future  workshops  and  build 
faculty  strength  to  meet  future  demand.  While  programs  have 
already  been  scheduled  forthe  balance  of  Fiscal  Year  (FY)  2017, 
we  expect  that  FY  2018  will  offer  opportunities  for  expanding 
the  workshop  to  many  more  programs  across  the  Army  and 
other  DoD  acquisition  organizations.  Depending  upon  demand, 
a  lengthy  lead  time  might  be  needed  to  schedule  this  new  work¬ 
shop  as  added  DAU  faculty  members  ramp  up.  This  constraint 
suggests  early  engagement  and  discussion  of  availability  with 
the  appropriate  DAU  Regional  representatives. 

How  is  this  training  fun?  The  fun  part  is  actually  working  in 
small  teams  and  applying  critical  thinking  skills  to  solving  real 
program  problems.  Indeed,  the  workshop  may  provide  the  first 
occasion  for  working  as  a  team  in  an  environment  where  team 
members  can  singularly  focus  on  developing  their  acquisition 
strategies  without  distractions.  The  facilitators  have  observed 
that  the  teams  enjoy  rising  to  the  challenge  and  thinking  about 
their  issues  with  new  tools  and  methods.  It  can  be  exciting  for 
them  to  develop  new  approaches  and  ideas  that  they  hadn't 
thought  of  or  used  before.  Studies  show  that  the  energy  level 
and  enthusiasm  of  small  teams  can  be  significant,  given  the 
right  conditions,  including  well-established  team  norms  of  be¬ 
havior,  respect  for  everyone's  input,  emotional  intelligence, 
and  other  traits  often  associated  with  soft  skills. 

The  workshop's  critical  thinking  module  on  the  first  day  sets 
a  baseline  for  this  type  of  team  interaction  and  is  reinforced 
throughout  the  week.  To  increase  this  emphasis,  a  module 
on  high-performing  teams  has  been  added  to  the  agenda  on 
the  last  day.  It  culminates  with  a  team  survival  exercise  that 
challenges  participants  to  excel  in  group  problem  solving  and 
collaboration. 

In  closing,  we  believe  that  this  new  workshop  offers  great  po¬ 
tential  to  assist  teams  in  planning  their  new  acquisition  strat¬ 
egies.  While  the  initial  feedback  has  been  positive,  we  need 
to  follow  up  with  the  teams  several  months  later  to  evaluate 
the  training's  effect  on  program  outcomes,  products,  and/or 
processes.  This  longer-term  evaluation  will  enable  us  to  make 
subsequent  workshop  improvements  and  assist  us  in  deter¬ 
mining  what  tools  and  practices  best  help  program  or  project 
managers  and  their  teams  to  succeed.  & 


The  authors  can  be  contacted  at  brian.schultz@dau.mil  and 
jeffrey.w.megargel.ctr@mail.mil. 
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How  to  Improve 
DoD-Industry  Collaboration 

Mandate  It  and  Make  It  a  Mindset 


Lt.  Col.  Douglas  B.  Ogden ,  USMC 


For  the  past  several  years,  the  government  has  directed  that  innovation  be 
made  a  key  ingredient  of  acquisition.  Industry  has  embraced  this  idea,  and 
it  has  become  a  standard  practice.  Many  Department  of  Defense  (DoD) 
suppliers  have  stood  up  innovation  cells  within  their  organization,  whereas 
others  have  claimed  that  their  entire  organization  is  the  innovation  cell. 


So  what  other  practices  can  we  embrace  in  order  to  improve  acquisition?  What  should  we  be  doing  more  of  in 
order  to  develop  and  deliver  quality  products,  on  time,  at  an  affordable  price?  One  thing  we  can  do  is  ensure 
we  maximize  collaboration  between  all  stakeholders  when  developing  new  requirements,  designing 
new  products,  or  resolving  issues  on  existing  programs. 


tiltrotors.  He  wrote  this  article  while  serving  on  a 
^  Secretary  of  Defense  Corporate  Fellow¬ 
ship  in  2016. 


Ogden  is  a  U.S.  Marine  Corps  Acquisition  Professional  with  13  years  of  acquisition 
experience,  most  of  which  has  been  in  program  management.  He  has  served 
as  the  platform  Integrated  Product  Team  Lead  for  H-53E  helicopters, 
VH-3D  and  VH-60N  Presidential  helicopters,  and  MV-22B 


Defense  AT&L:  July-August 


Requirements  Generation 

It  is  widely  accepted  that  successful  programs  start  with  solid, 
stable,  achievable  requirements.  So  we  should  involve  every 
organization  that  can  provide  a  benefit  generating  those  re¬ 
quirements.  This  would  include  not  only  all  of  the  government 
agencies  to  be  affected  by  the  requirements  but  also  our  in¬ 
dustry  partners.  There  are  many  past  examples  where  we  have 
developed  requirements  without  talking  to  industry,  only  to 
find  out  after  a  contract  was  awarded  that  we  were  "shooting 
for  the  moon"  and  challenging  industry  too  hard.  The  next 
thing  you  knew,  we  were  painfully  negotiating  with  the  con¬ 
tractor  to  change  the  contract,  adding  time  to  the  schedule  and 
increasing  the  program  cost.  All  of  this  waste  of  time  and  effort 
could  have  been  avoided  if  we  had  collaborated  up  front  and 
jointly  developed  reasonable  and  achievable  requirements. 


Distributing  draft  requirements  out  to  all  stakeholders,  includ¬ 
ing  industry,  should  be  the  standard  procedure.  The  Air  Force 
is  actively  doing  this  as  part  of  their  "Bend  the  Cost  Curve" 
initiative.  The  primary  goal  is  to  link  government  and  indus¬ 
try  together  early  and  often  in  the  acquisition  process  to  set 
realistic  expectations,  develop  achievable  requirements,  and 
ensure  that  we  receive  our  products  in  a  timely  manner  and  at 
an  affordable  price.  An  example  of  this  initiative  in  work  is  the 
Air  Force  T-X  competition.  As  the  Air  Force  developed  require¬ 
ments  for  an  aircraft  to  replace  the  aging  T-38  trainer,  draft 
versions  were  distributed  to  industry  to  allow  for  open  and 
deliberate  dialogue.  The  benefits  of  this  approach 
are  significant: 


Designing  New  Products 

When  designing  new  products,  the  goal  should  always  be  the 
achievement  of  acceptable  quality  the  first  time  around.  In  an 
ideal  world,  every  product  designed  would  require  no  rework 
or  redesign  when  the  prototype  is  fabricated.  While  that  may 
be  hard  to  achieve,  it  should  be  the  goal.  The  closer  we  can 
get  to  that  ideal  condition,  the  more  time  and  money  we  save, 
ultimately  getting  the  product  to  the  customer  as  quickly 
as  possible  and  at  the  best  price  possible.  So  how  do  we  do 
that?  By  involving  all  required  stakeholders  in  the  design  and 
development  process.  Today's  weapons  systems  are  more 
complex  than  ever  and,  given  current  budget  constraints, 
program  execution  has  become  increasingly  difficult.  Having 
a  strong  industry/government  team  is  more  imperative 
than  ever.  Making  the  investment  up  front  to 


•  Contractors  can  inform 
the  government 
of  their 


true 

capabilities, 

and  their  limitations,  in¬ 
formation  that  can  help  the  gov¬ 
ernment  develop  achievable  requirements. 

Often  the  government  isn't  fully  aware  of  all  the 
developments  within  industry.  If  two-way  dialogue  is  initi¬ 
ated  between  government  and  industry  while  the  require¬ 
ments  are  still  being  drafted,  we  can  develop  reasonable 
requirements  that  have  a  better  chance  of  being  achieved. 
Contractors  can  develop  better  proposals.  If  industry  fully 
understands  what  the  government  wants,  it  can  focus  and 
maximize  its  innovations  to  develop  higher-quality,  timely 
and  affordable  solutions  with  a  greater  chance  of  success. 
Contractors  are  much  more  apt  to  enter  into  fixed  price  con¬ 
tracts  if  they  are  involved  in  the  requirements  development 
up  front.  They  will  be  more  comfortable  with  their  plan,  and 
be  more  willing  to  accept  more  of  the  risk. 
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assemble  the  government  requirement  owners,  government 
program  office  personnel,  and  industry  to  work  together  on 
the  design  of  the  product  will  ensure  the  best  chance  of 
achieving  first-time  quality. 

Issue  Resolution 

Most  would  agree  that  the  best-laid  plans  never  survive  first 
contact  with  reality.  Issues  will  arise  on  every  program  we 
develop,  some  of  them  causing  significant  schedule  slips  and 
cost  growth  when  they  are  not  dealt  with  promptly  and  intelli¬ 
gently.  Fortunately  for  us,  our  government  and  industry  teams 
include  a  diverse  group  of  people,  with  each  person  bringing 
different  experiences,  knowledge  and  capabilities  to  the  table. 
When  only  one  person  works  on  solving  a  problem,  the  solu¬ 
tions  are  limited  to  that  person's  knowledge  and  experience. 
When  several  people  work  an  issue,  the  chances  of  arriving 
at  a  better  solution  increase.  Putting  more  heads  together  on 
a  problem  increases  the  odds  of  resolving  issues  faster,  better 
and  with  potential  cost  savings  or  cost  avoidance. 


We  all  must  realize  that  we  are  not  going  to  get  along  with 
everybody,  nor  are  we  going  to  please  everybody.  Consider 
it  a  challenge  to  develop  better  interpersonal  skills.  Keep  it 
professional  and  focus  the  project  on  the  warfighter  you  are 
ultimately  supporting;  remind  your  compadre  of  the  common 
goal.  If  a  working  relationship  cannot  be  forged,  then  it  needs 
to  be  taken  to  a  higher  level  for  resolution.  Nonfunctional  re¬ 
lationships  only  hamperthe  effort  and  ultimately  the  end  user. 

Lack  of  trust:  This  usually  builds  over  time  when  people  do 
not  get  along.  The  only  way  to  regain  or  build  trust  from  the 
ground  up  is  to  work  together  as  a  team  and  earn  it.  Again, 
moral  courage  is  required. 

Lack  of  transparency:  Some  people  resist  sharing  information 
that  may  highlight  issues,  focus  unwanted  attention  on  failures, 
and  highlight  who  is  responsible.  Fear  of  retribution  is  certainly 
natural,  but  bad  news  does  not  get  better  with  time. So  the 
sooner  we  can  solve  problems  and  eliminate  them,  the  better 
chance  we  have  of  keeping  the  project  on  track. 


Barriers  to  Collaboration 

So  if  we  know  collaboration  is  a  good  thing,  why  don't  we  do 
more  of  it?  Because  there  are  barriers  to  collaboration  that  we 
must  address  and  resolve.  Some  examples: 

Egos:  In  a  profession  filled  with  "Type  A''  personalities,  there 
are  glory  hounds  who  want  to  take  all  of  the  credit  for  success 
and  not  involve  others  in  solving  problems.  Acquisition  profes¬ 
sionals  must  check  their  egos  at  the  door  and  remember  that 
acquisition  is  a  team  sport. 

Fear  of  working  in  a  group:  I  n  contrast  to  the  Type  A's,  there 
are  introverts  in  the  business  who  prefer  to  work  alone  or  are 
embarrassed  to  be  in  groups  for  fear  that  they  may  show  a 
lack  of  knowledge  or  say  something  wrong  in  a  meeting.  These 
folks  must  embrace  their  fear  and  conquer  it.  They  are  not 
helping  the  collective  effort  by  working  in  a  silo. 


Conflicti 

mc 


Conflicting  personalities:  Not  everybody  gets  along,  and 
most  of  us  have  at  one  time  or  another  felt  we  knew 
whom  we  could  talk  to  in  order  to  resolve  an 
n  issue,  but  cringed  at  the  thought  of  call¬ 
ing  them  because  we  just  didn't  get 
along  with  them.  Breaking 
through  this  reluctance 
takes  some  moral 
i  courage. 


If  two-way 
dialogue  is  initiated 
between  government  and 
\industry  while  the  requirement?^ , 

\  ifc  |  i  j 

are  still  being  drafted,  we  can  develop 


Limited  travel  budget:  Current  budget  constraints  mean  that 
fewer  people  are  allowed  to  travel.  This  is  unfortunate,  be¬ 
cause  most  people  prefer  face-to-face  collaboration  for  several 
reasons.  Body  language  can  be  read,  and  no  one  can  hit  the 
"mute"  button  to  have  a  private  sidebar  without  the  rest  of  the 
team  hearing.  Because  of  this,  transparency  is  better  achieved 
in  face-to-face  meetings.  Also,  you  don't  have  the  problem  of 
more  than  one  person  trying  to  talk  at  the  same  time,  which 
often  occurs  during  telecoms  or  Web-based  program  events 
and  can  lead  to  frustration.  Finally,  in-person  meetings  help 
form  relationships  and  team  cohesion.  However,  our  reduced 
budget  has  increased  the  need  for  use  of  electronic  collabora¬ 
tive  tools  in  order  to  connect  team  members  around  the  globe. 

Difficulty  with  electronic  collaboration  tools:  Some  people 
are  hesitant  to  use  collaboration  tools  such  as  video  teleconfer¬ 
ence  (VTC)  or  Web-based  meeting  programs  because  of  the 
effort  required  to  coordinate  an  electronic  meeting,  the  diffi¬ 
culty  in  using  the  available  tools,  the  poor  quality  of  the  tools 
(delayed  video  and  audio,  etc.),  or  because  not  every  team 
member  has  access  to  the  tool.  This  merits  further  discussion. 

Electronic  Collaboration  Tools 

There  are  many  collaboration  tools  out  there,  ranging  from  low 
fidelity  and  inexpensive,  to  high  fidelity  and  very  expensive. 
Each  has  different  capabilities  and  its  own  advantages  and 
disadvantages.  The  electronic  tools  chosen  for  your  program 
should  be  tailored  to  the  size  and  complexity  of  your  project— 
and,  obviously,  your  budget.  In  several  interviews  with  both 
government  and  industry  employees,  some  common 
themes  emerged: 


a  better  chance  of  being  achieved. 
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•  Collaboration  improves  relation¬ 
ships  and  increases  produc¬ 
tivity. 
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■  Good 
collabora¬ 
tion  can  help  pri 
oritize  requirements  and 
prevent  faulty  assumptions. 

■  Face-to-face  is  best,  but  it's  hard  to  get 
all  the  players  in  one  place. 

■  Collaboration  tools  in  some  form  typically  are  avail 
able  in  their  workspace,  whether  at  a  computer  or  in  a 
conference  room,  or  both. 

■  Not  all  collaboration  tools  function  well.  Sources  of  frus¬ 
trations  include  that  the  tools  are  difficult  or  cumbersome 
to  set  up;  they  are  not  intuitive;  their  video,  audio,  or  dis¬ 
play  of  documents  and  briefs  are  poor;  and  not  all  players 
have  access  to  the  collaboration  tool. 

Understanding  and  dealing  with  these  complaints  is  criti¬ 
cal,  because  your  team's  constant  frustration  with  the  tools 
provided  will  inhibit  collaboration  and  not  provide  the  benefit 
sought.  Therefore,  in  choosing  your  team's  electronic  tools, 
consider  quality,  ease  of  use,  ease  of  setting  up,  accessibility 
to  all  team  members  and  stakeholders,  and  affordability. 


Collaboration  is  more  than  just  getting  together;  it 
is  working  together.  Having  regularly 
scheduled  meetings  or  telecbmfV 
be  more  than  just  a  status  ca 
Meetings'mSftfo  bf 


would  pres¬ 
ent  an  issue. 

Over  time,  my  auto¬ 
matic  response  to  this  be¬ 
came,  "Let's  determine  the  Root 
Cause,  then  develop  and  implement  cor¬ 
rective  actions."  Collaboration  needs  to  become 
an  automatic  response  in  much  the  same  way.  When¬ 
ever  a  team  leader  or  manager  is  presented  with  an  issue,  or 
a  capability  gap,  the  automatic  response  should  be  "Who  are 
we  getting  together  to  solve  this,  and  how  are  we  going  to  get 
them  together."  Make  it  a  mindset. 


productive 


Recommendations 

The  center  of  the  universe  for  acquisition  programs  is  the  pro¬ 
gram  manager  (PM),  who  is  accountable  for  the  program's 
success  or  failure.  Therefore,  it  is  the  PM's  responsibility  to 
lead  the  team,  ensure  balanced  collaboration,  and  continually 
drive  the  team  to  accomplish  the  mission  in  the  most  timely 
and  affordable  manner. 

PMs  cannot  run  programs  all  by  themselves.  They  rely  on  their 
team  of  professionals  and  all  of  the  stakeholders  to  contribute 
to  their  programs'  success.  Therefore,  they  must  be  the  driving 
force  to  ensure  that  everyone  works  together  effectively  and 
efficiently.  Not  everyone  is  wired  to  naturally  want  to  work 
with  others,  so  many  team  members  must  be  guided  by  their 
leaders  and  managers.  Government  PMs  must  focus  on  set¬ 
ting  up  and  maintaining  good  government-to-government  rela¬ 
tionships,  and  government-to-industry  relationships.  Industry 
PMs  must  focus  on  setting  up  and  maintaining  good  industry- 
to-government  relationships,  and  industry-to-industry  rela¬ 
tionships  (including  those  with  both  prime  contractors  and 
subcontractors).  This  is  easier  said  than  done  because  not 
all  government  agencies  agree  with  each  other,  and  industry 
primes  often  see  their  subcontractors  or  industry  partners  as 
"competimates."  The  only  way  to  break  through  these  differ¬ 
ences  and  build  trust  and  relationships  across  the  board  is 
strong  leadership  in  both  government  and  industry. 

So  how  do  we  guide  our  teams  to  work  together  and  break 
through  the  barriers?  First  of  all,  we  need  to  make  working  to¬ 
gether  a  mindset.  As  an  example,  in  my  previous  roles  as  an  In¬ 
tegrated  Product  Team  Lead  on  acquisition  programs,  I  devel¬ 
oped  an  almost  Pavlovian  response  to  issues  when  they  arose. 
Almost  every  day,  someone  from  my  team  in  the  program  of¬ 
fice,  or  a  stakeholder  from  the  fleet  or  the  Pentagon  or  industry, 


The  second  part  of  the  answer  is  to  mandate  collaboration. 
Every  team  typically  has  a  charter  that  establishes  roles  and  re¬ 
sponsibilities,  so  why  not  include  in  that  charter  a  collaboration 
plan?  It  need  not  be  elaborate,  but  should  simply  state  how 
and  when  the  stakeholders  will  get  together— weekly,  monthly, 
quarterly,  in  person,  by  telecom,  by  Web-based  program,  by 
VTC,  etc.  If  you  want  your  team  to  work  together,  you  have  to 
organize  it  and  make  it  happen. 

The  third  part  of  the  answer  is  to  provide  the  necessary  tools 
to  allow  the  collaboration.  This  includes  providing  funding  for 
people  who  must  travel  when  face-to-face  meetings  are  re¬ 
quired,  and  also  providing  the  appropriate  level  of  effective 
electronic  tools  so  the  team  can  collaborate  with  stakeholders 
around  the  world  at  the  click  of  a  button.  Significant  savings 
can  be  realized  by  having  a  Web-based  meeting  or  VTC,  as 
opposed  to  flying  in  people  from  faraway  places. 

Selection  of  the  appropriate  electronic  tools  will  depend  on  the 
size  and  range  of  the  team,  the  complexity  of  the  product  being 
developed,  and,  obviously,  the  budgetary  limitations.  The  tools 
must  be  easy  to  set  up,  accessible  to  all  stakeholders,  of  good 
quality— and  their  use  must  be  intuitive.  If  the  tools  are  difficult 
to  set  up,  no  one  will  take  the  time.  If  not  all  stakeholders  can 
access  the  tools,  the  team  will  not  be  completely  involved.  If 
the  tools  are  of  poor  quality  or  difficult  to  use  and  understand, 
no  one  will  want  to  use  them. 

Another  recommendation  is  to  make  each  of  your  conference 
rooms  Collaboration  Centers.  A  conference  room  is  simply  a 
room  with  tables,  chairs  and  a  drawing  board.  A  Collabora¬ 
tion  Center  has  collaboration  capability  such  as  a  computer 
interface  that  provides  a  Web-based  program  or  VTC,  large 
monitors  that  can  display  documents,  briefs  and  drawings, 
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and  cameras  that  allow  all  players  to  see  each  other.  Once 
you  select  or  design  a  Collaboration  Center  that  is  appropri¬ 
ate  for  your  program  size  and  complexity,  strive  to  have  all  of 
your  Collaboration  Centers  in  your  program  office  of  the  same 
configuration.  Having  the  same  procedures  and  capabilities  in 
each  room  will  reduce  frustration  on  your  team  and  simplify 
the  support  of  information  technology. 

The  final  PM  challenge  is  to  ensure  balanced  collaboration. 
Bringing  people  together  needs  to  be  for  meaningful  work  that 
will  contribute  to  moving  the  ball  down  the  field.  Collaboration 
is  more  than  just  getting  together;  it  is  working  together.  Hav¬ 
ing  regularly  scheduled  meetings  or  telecoms  must  be  more 
than  just  a  status  call.  Meetings  need  to  be  productive.  This  is 
where  leadership  comes  back  into  play  again.  A  team  leader 
needs  to  ensure  that  the  team  is  truly  working  together  and 
progressing  toward  a  common  goal. 

You  also  must  be  careful  not  to  over-collaborate.  Not  everyone 
needs  to  collaborate  with  everyone.  Over-collaborating  can 
burn  people  out,  and  can  be  a  source  of  team  consternation. 
So,  in  addition  to  holding  only  meetings  that  have  a  purpose, 


make  sure  only  the  required  personnel  attend  the  appropriate 
meetings.  Likewise,  encourage  your  team  members  to  excuse 
themselves  so  they  can  continue  with  their  work  if  they  feel 
they  do  not  need  to  attend  a  particular  meeting. 

A  final  recommendation  is  to  provide  a  Web-based  col¬ 
laboration  forum  to  your  team  members  to  allow  them  to 
communicate  and  solve  problems  together.  Many  issues 
have  been  solved  before,  and  reinventing  the  wheel  repeat¬ 
edly  is  just  wasteful.  A  Web-based  forum  would  allow  your 
team  to  post  issues  that  others  may  have  already  resolved, 
pass  on  success  stories  that  others  can  learn  from,  and 
build  team  cohesion. 

There  are  many  ingredients  in  building  a  highly  functioning 
team  that  runs  like  a  well-oiled  machine,  but  it  all  comes  down 
to  leadership.  Good  leaders  listen  to  their  teams,  gain  an  un¬ 
derstanding  of  what  their  teams  need  to  execute,  and  then 
provide  their  teams  with  the  appropriate  tools  to  accomplish 
their  mission  and  consistently  produce  great  results.  & 

The  author  can  be  contacted  at  douglas.ogden@navy.mil. 
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DAU  is  continually  looking  for  new  topics,  tools  and  resources  to  help  you  succeed  on  the  job. 

HOW  DO  WE  KNOW  WHAT  YOU  NEED? 

iL  i  YOU  TELL  US! 


VISIT 


https://www.dau.miI/tools/t/Request-a-New-Tool 

Take  a  quick  two-question  survey  on  the  topics,  tools,  and  resources  you  need. 
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The  Ethical  Imperative 
and  Courage  to  Cancel 


Eugene  A.  Razzetti 


Looking  back  on  an  association  of  more  than  40  years 
with  the  military  (27  years  spent  on  active  duty)  and 
a  more  than  50-year  career  in  management,  including 
time  at  the  Pentagon;  and  then  (most  recently)  reading 
the  farewell  writings  of  Under  Secretary  of  Defense  for 
Acquisition,  Technology,  and  Logistics  Frank  Kendall,  I  am 
forced  unenthusiastically  to  several  conclusions: 


Razzetti,  a  retired  U.S.  Navy  captain ,  is  a  management  consultant  and  military  analyst.  He  au¬ 
thored  five  management  books ,  including  The  Executive's  Guide  to  Corporate  Responsibility 
Management  and  MVO  8000  and  has  served  on  the  advisory  boards  of  two  business  schools. 
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The  Ethical 
Imperative: 

Lives  are  at  stake— 
either  make  the 
program  work  or 
make  it  go  away! 
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My  decision  briefing  to  my  bosses  at  CNA  ended  with 
a  slide  bullet  that  read:  "We  have  an  Ethical  Imperative 
to  withdraw  our  support  from  the  program/'  I  had  never 
thought  to  use  that  term  before. 


■  Some  Department  of  Defense  (DoD)  acquisition 
problems  come  from  Congress  and  can  thwart  the  best 
efforts  of  the  most  conscientious  program  managers. 

■  The  same  practices  and  procedures  that  characterize 
the  successful  management  of  acquisition  programs 
can  (and  should)  lead  unapologetically  to  a  decision  to 
cancel  those  same  programs. 

■  Technical  and  programmatic  problems,  sooner  or  later, 
have  legal  and  ethical  implications. 

In  1982,  as  a  young  commander  in  the  Pentagon  and  earlier 
having  squandered  my  Navy  career  at  sea,  I  tried  to  kill  an 
overly  bloated  program  for  a  mine  countermeasures  system. 
The  program  was  devoid  of  meaningful  milestones,  the  con¬ 
tract  devoid  of  meaningful  sanctions,  and  the  vision  devoid 
of  meaningful  reality.  The  delivered  product  would  not  have 
performed  as  required  and  sailors  relying  on  it  would  have 
been  endangered.  The  only  indicators  trending  upward  for 
the  bloated  program  were  its  cost  and  the  completion  date. 
The  Chief  of  Naval  Operations  wanted  to  kill  it.  Funds  were 
tight,  and  we  had  many  smaller,  more  cost-effective,  "'bread 
and  butter"  programs  that  could  have  used  those  funds.  We 
wanted  to  put  the  funds  to  better  use,  for  predictable  and 
measurable  benefits. 

The  prime  contractor,  who  had  established  a  plant  in  the 
home  state  of  a  member  of  the  Senate  Armed  Services  Com¬ 
mittee,  complained  to  the  senator.  Predictably,  the  senator 
directed  that  we  put  the  funding  back.  Funding  for  the  sen¬ 
sible  but  unglamorous  programs  was  not  his  concern. 

In  2006,  as  a  military  analyst  for  the  Center  for  Naval  Analyses 
(CNA),  I  worked  on  a  program  intended  to  develop  sensors 
for  detecting  chemical  and  biological  agents  and  to  deploy 
those  sensors  in  strategic  ports  of  embarkation/debarkation 
(in  the  Middle  East).  The  program  was  literally  a  "401k"  for 
contractors,  systems  commands  and  "scientists"  of  all  types. 
But  it  potentially  endangered  the  lives  of  soldiers,  sailors  and 
Marines.  Field  testing  results  were  deceptive,  and  contractors 
tried  lowering  the  bar  to  skew  test  results  in  their  favor. 

My  decision  briefing  to  my  bosses  at  CNA  ended  with  a  slide 
bullet  that  read:  "We  have  an  Ethical  Imperative  to  withdraw 
our  support  from  the  program."  I  had  never  thought  to  use 


that  term  before.  Management  agreed,  and  we  helped  to 
drive  a  stake  into  the  heart  of  a  shameless,  wasteful  and  po¬ 
tentially  dangerous  program— in  the  meantime  saving  our 
own  reputations. 

The  Ethical  Imperative  and  the  Best 
Guidance  You  Could  Ever  Want 

When  Adm.  James  D.  Watkins  was  tapped  to  return  to 
Washington  as  Chief  of  Naval  Operations,  he  brought  back 
with  him  (then)  Vice  Adm.  Lee  Baggett,  Jr.,  to  become  Di¬ 
rector  of  Naval  Warfare.  Watkins'  guidance  to  Baggett  to 
improve  performance  of  programs  in  the  contentious  and 
often  antagonistic  branch  was:  "Either  make  it  work  or  make 
it  go  away."  Unsurprisingly,  Baggett  made  it  work.  Watkins' 
guidance  had  a  profound  influence  not  only  on  an  important 
branch  of  the  Navy,  but  also  on  an  unimportant  young  com¬ 
mander  in  the  Pentagon's  trenches.  "Make  it  work  or  make  it 
go  away."  What  wiser  guidance  could  ever  be  given,  and  what 
more  welcomed  guidance  could  ever  be  received? 

Ethics  in  Program  Management 

For  our  purposes,  consider  "ethics"  as  a  systematic  reflec¬ 
tion  on  rules  and  issues— the  way  people  act  and  the  rules 
that  form  the  basis  of  their  actions.  Most  of  us  endeavor  to 
do  what  we  believe  is  right.  To  drive  safely  for  the  benefit  of 
pedestrians  is  ethical;  to  drive  safely  because  it  is  the  law 
is  to  fear  a  penalty  and/or  punishment.  Ethics  come  from 
within  a  person's  moral  sense  and  desire  to  preserve  his  or 
her  self-respect.  Legal  requirements  come  from  outside  the 
individual.  Organizational  ethics  (which  I  call  Corporate  Re¬ 
sponsibility  Management)  institutionalize  reflection  on  those 
rules  and  issues,  and  then  create  and  control  all  of  its  pro¬ 
cesses  to  ensure  that  the  organization  in  question  performs 
to  established  ethical  standards. 

To  operate  "legally"  is  to  operate  in  conformance  with  all 
established  laws  and  regulations  applicable  to  the  mission  of 
the  organization.  Laws  and  regulations  are  created  by  govern¬ 
ments  to  protect  their  citizens. 

Table  1  describes  the  workaday  preoccupations  of  program 
managers  and  staffs.  Since  (theoretically  at  least)  our  laws 
reflect  our  ethical  beliefs,  you  can  adhere  to  both  and  violate 
both  simultaneously.  An  ethics-centered  organization  has  an 
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Table  1.  Legal  and  Ethical  Problem  Areas 
in  Program  Management 


Problem  Area 

Legal 

Ethical 

Vision  and  Mission  Development 

v 

✓ 

Strategic  Planning 

s 

v 

Financial  Management 

s 

✓ 

Contract  Management 

s 

■/ 

Technology 

s 

Configuration  Management 

■/ 

■/ 

Research  and  Development 

s 

Risk  Management 

s 

•/ 

Supply  Chain  Management 

V 

s 

Data  Analyses 

s 

s 

Training  and  Qualification 

V 

s 

Vendor  Management 

V 

V 

Tables  and  figures  by  the  author 


organizational  "character”  and  strategy,  taking  very  seriously 
not  only  its  mission  but  also  its  responsibility  to  employees, 
customers  and  the  community. 

Ethical  violations  may  not  result  in  punishment;  legal  vio¬ 
lations  carry  punishment  to  the  violators.  When  program 
managers  uncover  legal  problems,  they  create,  de  facto,  an 
ethical  imperative  to  take  corrective  action.  Accordingly,  it 
is  virtually  impossible  to  have  only  a  legal  problem  or  only 
an  ethical  problem. 

Critical  Thinking  and  Sound  Judgment 

In  his  final  article  in  the  January-February  2017  issue  of  De¬ 
fense  AT&L,  Kendall  also  wrote  about  the  need  for  critical 
thinking  and  sound  professional  judgment.  To  that,  we  now 
add  Ethical  Imperative  and  create  what  you  could  call  the 
three  musketeers  of  program  management.  Like  the  slogan 
of  Alexandre  Dumas'  courageous  "Three  Musketeers”  of  old: 
"One  for  all  and  all  for  one.” 

Critical  thinking,  according  to  Wikipedia,  demands  the  abil¬ 
ity  to: 

■  Recognize  problems,  and  find  workable  means  for  meet¬ 
ing  those  problems. 

■  Understand  the  importance  of  prioritization  and  order  of 
precedence  in  problem  solving. 

■  Gather  and  marshal  pertinent  (relevant)  information. 

■  Recognize  unstated  assumptions  and  values. 

■  Comprehend  and  use  language  with  accuracy,  clarity 
and  discernment. 


■  Interpret  data,  to  appraise  evidence  and  evaluate 
arguments. 

■  Recognize  the  existence  (or  nonexistence)  of  logical 
relationships  between  propositions. 

■  Draw  warranted  conclusions  and  generalizations. 

■  Test  one's  conclusions  and  generalizations. 

■  Reconstruct  one's  patterns  of  beliefs  on  the  basis  of 
wider  experience. 

■  Render  accurate  judgments  about  specific  things  and 
qualities  in  everyday  life. 

In  sum:  "Critical  thinking  is  a  persistent  effort  to  examine 
any  belief  or  supposed  form  of  knowledge  in  the  light  of  the 
evidence  that  supports  or  refutes  it  and  the  further  conclu¬ 
sions  to  which  it  tends.” 

Sound  professional  judgment  means  our  capacity  to  assess 
situations  or  circumstances  wisely  and  to  draw  sound  con¬ 
clusions.  This  involves  applying  relevant  training,  knowledge 
and  experience  within  the  context  provided  by  pertinent  pro¬ 
fessional  and  technical  standards,  as  applicable,  in  making 
informed  decisions  about  appropriate  courses  of  action. 

Outside  auditors  (like  me)  are  reminded  continually  of  the 
requirement  under  international  standards  such  as  Interna¬ 
tional  Standards  Organization  9000  to  apply  a  structured 
skepticism  throughout  the  audits.  Auditing,  in  this  sense,  is 
institutionalized  and  structured  critical  thinking  and  refers  to 
the  appropriate  application  of  professional  skepticism. 

Vision  vs.  Reality 

Another  part  of  Kendall's  article  concerned  vision  vs.  reality 
and  how  important  it  is  for  a  product  or  system  to  address 
the  threats  and  employ  the  capabilities  of  the  real  world. 

Figure  1  describes  the  transition  of  visions  and  missions  into  ac¬ 
tionable  systems  and  equipment  through  a  series  of  sound  pro¬ 
gram  management  processes.  It  also  serves  to  remind  the  reader 
that  conformity  with  sound  management  processes,  de  facto,  cre¬ 
ates  an  optimal  environment  for  sound  ethical  decision  making. 

Requirements  analysis,  for  our  purposes,  means  determin¬ 
ing  the  needs  or  conditions  to  meet  new  or  altered  project 
requirements.  Robust  requirements  analyses  are  essential 
to  the  success  of  a  DoD  acquisition  program.  The  structure 
and  outcome  of  the  analyses  must  be  documented,  mea¬ 
surable  and  capable  of  being  verified  and  validated.  Most 
importantly,  the  analyses  must  create  actionable  intelligence 
related  to  program  needs  and  opportunities.  Analyses  must 
be  clear,  complete,  consistent  and  unambiguous.  The  find¬ 
ings  of  the  (needs  and)  requirements  analyses  form  the  heart 
and  soul  of  the  contract,  especially  as  reflected  in  the  design, 
statement  of  work,  and  all  the  measurable  goals  and  objec¬ 
tives.  Milestones  dates  must  be  enforceable  and  enforced. 

Due  diligence  can  be  defined  as  "investigation  by  or  on 
behalf  of  an  intended  buyer,  to  check  that  the  offeror  has 
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The  findings  of  the  (needs  and)  requirements  analyses 
form  the  heart  and  soul  of  the  contract,  especially  as 
reflected  in  the  design,  statement  of  work,  and  all  the 
measurable  goals  and  objectives. 


the  desired  assets,  capabilities,  technologies,  brand  rights, 
contracts,  and  other  attributes  required  by  the  buyer  and 
claimed  as  provided  by  the  offeror.”  Put  another  way,  all 
the  facts  are  available  and  have  been  verified.  Areas  for 
vendor  due  diligence  audit  and/or  verification  include  (to 
mention  the  big  ones): 

■  Environmental  and  occupational  safety  and  health 
compliance 

■  Corporate  security  (including  cyber  and  information 
systems  security) 

■  Manufacturing  quality  and  competence 

■  Adherence  to  federal  and  state  laws  and  regulations 

■  Budget  monitoring  and  controls 

■  Supplier  and  supply  chain  management 

■  Code  of  ethics  and  standards  of  conduct 


■  The  ability  of  investigate  root  causes  of  failures 

■  A  robust  set  of  internal  controls 

■  Engineering  and  configuration  management 

■  Capacity  for  growth  or  expansion. 

A  DoD  decision  to  make  a  major  purchase,  acquisition  or 
investment  should  reflect  a  comprehensive  and  structured 
due  diligence  process,  custom-tailored  for  the  specific  un¬ 
dertaking.  Regardless  of  the  outcome,  you  will  have  given  it 
your  best  shot. 

Program  Essentials 

Table  2  lists  fundamental  acquisition  program  management 
requirements.  Strength  in  all  of  them  is  essential;  weakness 
in  any  of  them  should  make  a  program  fair  game  for  critical 
review  and  possible  cancellation.  Moreover,  the  same  DoD 


Figure  1.  Vision  vs.  Reality 
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people  expected  to  manage  programs  successfully  should 
be  encouraged  to  make  their  voices  heard  when  cancellation 
is  indicated. 

The  last  row  is  the  “Ethical  Imperative."  The  program  is  sat¬ 
isfactory  (SAT)  if  management  recognizes  and  fosters  it  and 
unsatisfactory  (UNSAT)  if  it  does  not  exist. 

Bullet-proof,  Bullet-prone,  Bullet-worthy 

Programs,  either  at  their  inception  or  early  in  their  implemen¬ 
tation,  predictably  fall  into  one  of  three  categories: 

■  Bullet-proof  because  there  is  an  unimpeachable  vision, 
based  on  unimpeachable  needs  and  requirements  as¬ 
sessments,  and  they  contain  realistic  goals  and  action¬ 
able  objectives  and  milestones. 

■  Bullet-prone  because  the  vision  is  convincing,  but  does 
not  reflect  the  needs  and  requirements  assessments, 
and/or  the  metrics  are  lacking. 

■  Bullet-worthy  because  even  the  most  basic  criteria  de¬ 
scribed  above  are  not  measurably  being  met. 

Figure  2  illustrates  the  ongoing  necessity  to  monitor  pro¬ 
grams  and  to  act  forthrightly  when  action  is  necessary. 
Where  is  your  program  right  now,  and  why?  The  breakdown 
of  the  bars  is  for  illustration  only,  as  are  the  notional  areas 
being  measured.  The  points  to  remember  are: 

■  The  soldier,  sailor,  airman  or  Marine— not  the  member  of 
Congress— is  the  primary  stakeholder 

■  Constant  vigilance  is  the  order  of  the  day. 


Table  2.  DoD  Program  Essentials 


Program  Essentials 

SAT: 
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Manage  It 

Cancel  It 
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Milestones 
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Metrics 
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Vision  and  Mission  vs.  Reality 

s 

V 
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interference 

Contractor  competence/motivation 
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Congressional  involvement 
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Risk  Management  strategy 

s 
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Requirements  analyses 
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Configuration  management 

s 
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Supply  chain 
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Contract  structure  (Cost  Plus,  Fixed 

■/ 

Firm  Price) 

Contractor  Code  of  Ethics  and 

■/ 

■/ 

Standards  of  Conduct 

ETHICAL  IMPERATIVE- 

ALL  PARTIES 

Rounding  Up  the  Usual  Suspects 

Troubled  programs  don't  get  that  way  overnight.  The  signs 
are  there,  and  they  become  more  visible  with  each  missed 
milestone.  Program  managers  and  staffs  know  how  to  iden¬ 
tify  problems  but  often  have  difficulty  making  them  known 
to  (or  accepted  by) 


ably  determined  the  outcome.  This  does  not  mean  that  the 
program  manager's  findings  and  recommendations,  however 
quantifiable  or  actionable,  will  be  accepted  at  the  top  of  the 
totem  pole. 


higher  authorities, 
especially  if  the 
only  way  out  ap¬ 
pears  to  be  can¬ 
celing  the  program. 

It  is  likely  that 
many  of  the  prob¬ 
lems  discussed 
thus  far  will  be 
evident  in  a  trou¬ 
bled  program.  It  is 
just  as  likely  that 
program  manag¬ 
ers  can  show  in 
a  documentable 
way  when  some¬ 
thing  started  to  go 
wrong  and  predict- 
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Not  inappropriately,  the  decision  to  cancel  can  only  be 
made  if  there  is  the  courage  to  cancel. 


Table  3  includes  (cynically)  historical  causes  that  could  have 
delayed,  extended,  weakened  or  otherwise  trashed  DoD  pro¬ 
grams,  as  seen  by  the  outside  world.  Look  at  the  possible 
causes  and  see  if  you  can  find  the  right  one— and  good  luck 
proving  it. 

The  Courage  to  Cancel 

Not  inappropriately,  the  decision  to  cancel  can  only  be  made 
if  there  is  the  courage  to  cancel.  The  decision  to  cancel  is  the 
result  of  asking  questions  like  these— and  getting  the  wrong 
answer: 

■  Is  the  program  on  track  (with  sound  metrics,  milestones 
realistic/met)? 

■  Is  the  contractor  qualified  and  motivated,  or  did  it  just 
book  the  business? 

■  Have  all  the  legal  and  ethical  obligations  been  met? 

■  Does  the  program  "vision”  reflect  reality? 

■  Is  there  excessive  influence  from  stakeholders  and 
higher  authorities? 

■  Does  the  threat  of  protest  or  lawsuit  permeate  program 
operations? 

■  Is  the  program  risk  excessive  and/or  unmitigated? 

We  have  covered  the  justification  for  a  decision  to  cancel. 
The  courage  to  proceed  after  that  has  to  come  from  the  in¬ 
dividual  program  managers  and  their  chains  of  command. 
Take  courage  and  good  luck. 

Summary 

Put  this  sign  over  your  desk: 


The  only  thing  necessary  for  evil  to  triumph 
is  for  good  men  to  do  nothing . 

—  Edmund  Burke 


Table  3.  Outsider  View  of  What  Went 
Wrong 


Causes  of  Potential  Program  Failure  Can't  Be 
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Needs  analyses  inappropriate/incom-  •/ 
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✓ 


managers  uncover  legal  problems,  they  create,  de  facto,  an 
ethical  imperative  to  act.  It  is  virtually  impossible  to  have  only 
a  legal  problem  or  only  an  ethical  problem. 


The  effect  it  will  have  on  your  staff  and  your  visiting  contrac¬ 
tors  will  be  immediate  and  visible. 

It  was,  is,  and  will  always  be  great  sport  to  kill  a  program  that 
does  not  measure  up  to  requirements,  especially  when  it  is 
being  sustained  for  the  wrong  reasons. 

Faithfully  and  diligently,  program  managers  scrutinize  their 
processes.  In  doing  so,  they  predictably  either  reinforce  or 
expose  the  underpinnings  of  their  programs.  When  program 


The  same  measuring  rods  that  support  a  good  program  indict 
a  bad  one.  The  ethical  imperative— that  discomforting  re¬ 
minder  to  leaders  and  managers  that  when  lives  are  at  stake 
you  either  make  the  program  work  or  make  it  go  away— is 
both  timeless  and  universal. 

Simple?  Of  course!  But  why  then  is  it  applied  so  infrequently 
and  ignored  so  frequently— at  least  at  the  top?  & 


The  author  can  be  reached  at  generazz@aol.com. 
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John  McGregor  ■  Karen  Kostelnik  ■  David  Nelson 


Seventh  in  a  series  of  articles 


he  Office  of  Performance  Assessments  and  Root  Cause  Analyses  (PARCA) 
is  responsible  for  Earned  Value  Management  (EVM)  policy,  oversight  and 
governance  across  the  Department  of  Defense  (DoD). 


T 

To  be  effective,  EVM  practices  and  competencies  must  be  integrated  into  the  program  manager  (PM) 
acquisition  decision-making  process;  the  data  provided  by  the  EVM  System  (EVMS)  must  be  timely, 
accurate,  reliable  and  auditable;  and  the  EVMS  must  be  implemented  in  a  disciplined  manner  consistent  with  the 
32  Guidelines  in  the  Electronic  Industries  Alliance  Standard-748  EVMS  (EIA-748).  PARCA's  goal  is  to  increase 
earned  value's  constructive  attributes  for  DoD  contractor  firms  managing  acquisition  programs  by  reducing  the 
economic  burden  of  inefficient  EVM  implementation. 


One  of  PARCA's  core  tenets  in  accomplishing  that  mission  is  to  work  collaboratively  with  government  and  industry 
stakeholders  to  develop  and  publish  EVM  Policy.  PARCA  believes  that  policy  and  guidance  should  strike  the  ap¬ 
propriate  balance  between  cost  effectiveness  and  the  need  for  programmatic  situational  awareness.  There  is  a 


McGregor  is  the  Deputy  Director  for  Earned  Value  Management  in  the  Performance  Assessments  and  Root  Cause  Analyses  organization 
in  the  Office  of  the  Assistant  Secretary  of  Defense  for  Acquisition.  His  office  serves  as  the  Department  of  Defense  (DoD)  focal  point  for  all 
policy ,  guidance ,  and  competency  relating  to  Earned  Value  Management.  Kostelnik  and  Nelson  support  the  office  in  developing  policy  and 
interfacing  with  DoD's  government  and  industry  stakeholders. 
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natural  circle  of  EVM  policy  development  as  shown  in  Figure  1. 
This  cycle  reflects  the  fact  that  policies  sometimes  can  result 
in  unforeseen  consequences  that  require  updated  language. 

Drawing  on  the  experiences  of  both  industry  and  government 
experts  is  the  most  effective  method  of  ensuring  the  govern¬ 
ment's  requirements  are  affordable  and  achievable.  PARCA 
works  closely  with  military  Service  and  Agency  EVM  Focal 
Points,  the  Defense  Contract  Management  Agency  (DCMA), 
as  well  as  Industry  partners  led  by  the  National  Defense  In¬ 
dustry  Association  (NDIA)  Integrated  Program  Management 
Division  (IPMD),  to  ensure  that  policy  does  not  add  hardship 
but  facilitates  the  delivery  of  reliable  integrated  program  man¬ 
agement  data  for  decision  making. 

The  effective  application  of  EVM  policy  across  the  DoD  is  a  di¬ 
rect  outcome  of  PARCA's  collaborative  stakeholders  relation¬ 
ships.  As  examples  of  some  recent  PARCA  collaborations,  gov¬ 
ernment  and  industry  partners  provided  invaluable  feedback 
during  policy  updates  of  the  Integrated  Program  Management 
Report  ( IPMR )  and  associated  Guide,  and  the  EVM  System  Inter¬ 
pretation  Guide  ( EVMSIG )  that  provides  the  DoD  interpretation 
of  the  intent  of  the  EIA-748  EVMS  Guidelines.  The  enhance¬ 
ments  of  these  two  documents  clarify  the  EVM  system  and 
EVM  reporting  requirements.  Also,  PARCA  has  worked  over 
the  years  with  the  DoD  Joint  Space  Cost  Council  (JSCC)  in  its 
efforts  to  understand  the  government  and  industry  impacts 
and  benefits  of  EVM  and  EVMS  policies. 

Policymakers  should  be  willing  to  make  changes,  especially 
when  acquisition  methods  change.  One  example  is  the  in¬ 
creasing  use  of  Agile  Software  methodologies  in  DoD  acqui¬ 
sitions.  Even  though  EVM  principles  and  EVMS  requirements 
can  be  rigid,  there  is  some  flexibility  that  must  be  documented 
clearly  to  allow  for  its  understanding  and  dissemination.  In 
the  Agile  example,  PARCA  has  released  an  Agile  and  EVM 
guide  for  PMs  when  the  two  disciplines  are  used  together.  The 
guide  was  developed  through  a  yearlong  feedback  loop  with 
government  and  industry  including  EVM,  Information  Tech¬ 
nology,  and  Business  functional  constituents.  It  documents 
the  relationship  between  Agile  and  EVM  and  explains  areas 
of  flexibility  regarding  an  EVM  system's  setup. 

EVM  provides  a  disciplined  project  management  approach 
through  an  integrated  planning  and  control  system  for  autho¬ 
rized  work  in  order  to  achieve  cost,  schedule  and  technical 
performance  objectives.  EVM's  core  outcomes  are  meant  to 
provide  the  government  insight  into  contractor  performance, 
provide  actionable  information  and  help  ensure  confidence 
in  the  data  received.  There  are  several  keys  to  ensuring  that 
EVM  provides  value  to  government  and  industry  and  does  not 
drive  cost  into  the  system.  A  discussion  of  these  items  follows. 

Use  EVM  where  appropriate.  DoD  Instruction  5000.02  and 
Defense  Federal  Acquisition  Regulation  Supplement  (DFARS) 
Section  234.201  establish  dollar  and  contract-type  thresholds 
for  cases  when  EVM  should  be  utilized  and  incorporated  into 


Figure  1.  Policy  Development  Circle 
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a  contract.  In  addition  to  these  criteria,  PMs  should  consider 
work  scope  and  risk  in  deciding  whether  to  use  EVM  in  man¬ 
aging  their  programs.  EVM  is  most  beneficial  when  the  scope 
of  the  work  is  discrete  and  measurable.  Typical  types  of  work 
for  which  EVM  applies  include  development  and  integration 
efforts  with  specific  end  products  or  services  that  can  be  di¬ 
rectly  planned  and  measured.  It  may  be  appropriate  not  to 
apply  the  EVM  requirement  where  the  nature  of  the  work 
does  not  lend  itself  to  meaningful  EVM  information— such  as 
maintenance  and  repair  services  that  are  provided  as  needed. 
If  the  EVM  requirement  is  not  placed  on  a  contract  due  to  the 
nature  of  the  work,  the  PM  should  implement  an  alternative 
method  of  management  control  to  provide  advanced  warning 
of  potential  performance  problems. 

Understand  the  baseline  and  whether  it  is  achievable.  In 

order  to  evaluate  performance,  a  baseline  for  measuring  it 
should  be  established.  When  using  EVM  (or  when  managing 
any  program  without  EVM),  it  is  imperative  that  the  govern¬ 
ment  and  contractor  understand  the  work  scope  and  establish 
a  plan  to  achieve  technical,  cost  and  schedule  performance 
objectives.  An  Integrated  Baseline  Review  (IBR)  can  be  used 
as  a  means  for  government  and  contractor  to  discuss  the  work 
scope  and  the  plan  to  accomplish  the  contract's  statement 
of  work.  The  IBR's  outcome  should  be  that  the  government 
PM  understands  whether  the  baseline  is  achievable  in  view 
of  the  risk  profile.  An  IBR  begins  with  planning  the  baseline 
and  culminates  in  a  joint  review  event.  The  IBR  should  be  con¬ 
ducted  as  early  as  practical  for  the  program  within  the  required 
180  days  to  ensure  understanding  and  agreement  of  the  per¬ 
formance  measurement  baseline.  An  IBR  is  not  a  one-time 
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occurrence  and  can  be  held  at  any  time  throughout  the  con¬ 
tract  life  cycle  when  the  scope  of  work  changes. 

Manage  and  report  at  the  appropriate  level  of  detail.  Con¬ 
tractors  should  establish  a  work  breakdown  structure  (WBS) 
that  best  allows  them  to  manage  their  work  within  the  con¬ 
straints  of  their  systems.  The  level  of  detail  in  the  EVM  re¬ 
porting,  which  is  placed  on  contract  through  a  Contract  Data 
Requirements  List  referencing  the  IPMR,  should  be  based  on 


and  policies  to  a  wider  audience.  Office  representatives  have 
spoken  at  various  events  such  as  the  NDIA  IPMD  quarterly 
meetings,  EVM  World,  the  EVM  Integrated  Program  Man¬ 
agement  Workshop  (IPMW),  EVM  Practitioners  Symposium, 
Acquisition  Days  at  the  Defense  Acquisition  University,  and 
other  events.  PARCA  has  worked  diligently  to  maintain  two- 
way  communication  and  to  have  an  "open  door"  policy.  In 
fact,  PARCA  has  available  an  interpretation  process  for  any¬ 
one  to  ask  questions  for  clarification. 


Too  much  detail  in  reporting  will  slow  the  review,  response  and 
strategic  utility  of  the  data  and  potentially  detract  from  the  agile 
nature  needed  for  proactive  government  decision  making. 


scope,  complexity  and  risk.  Program  Offices  should  only  re¬ 
quire  EVM  data  deliverables  to  a  level  that  maintains  the  cost 
effectiveness  of  the  reporting,  targets  strategic  areas,  and 
aligns  with  how  the  program  is  managed.  Too  much  detail  in 
reporting  will  slow  the  review,  response  and  strategic  utility  of 
the  data  and  potentially  detract  from  the  agile  nature  needed 
for  proactive  government  decision  making. 

Use  the  EVM  data  to  make  decisions.  In  order  to  act  on  infor¬ 
mation  in  the  IPMR,  the  government  must  have  confidence  in 
the  data  provided.  The  EVMS  data  must  be  timely,  accurate, 
reliable,  auditable  and  implemented  in  a  disciplined  manner 
consistent  with  the  32  Guidelines  prescribed  in  EIA-748.  For 
programs  involving  more  than  $100  million,  where  EVM  is 
applied,  DCMA  reviews  the  system  for  compliance  and,  on 
an  ongoing  basis,  identifies  risks  to  ensure  the  reliability  of 
the  EVMS  data.  The  results  of  DCMA's  activities  are  vital  to 
the  largest  and  riskiest  DoD  programs.  Data  from  the  EVMS 
should  be  used  by  government  and  industry  stakeholders  to 
facilitate  joint  situational  awareness  and  in  making  their  deci¬ 
sions.  The  data's  value  is  in  what  can  be  predicted  and  acted 
upon  and  its  usefulness  in  leveraging  trend  analysis  of  past 
performance  and  identifying  the  root  cause  of  variances  from 
plan.  EVM  metrics  by  themselves  are  neither  good  nor  bad  but 
provide  data  points  and  information  as  a  basis  for  communica¬ 
tion  and  decision  making. 

In  its  mission  to  promote  disciplined  DoD  EVM  practices, 
PARCA  regularly  visits  DoD  installations,  Government  Ser¬ 
vice  Acquisition  Centers,  industry  facilities,  and  attends  EVM 
community  events  to  discuss  changes  to,  and  impacts  of, 
policy.  PARCA  provides  EVM  policy  training  and  initiative 
updates  at  industry  events  such  as  NDIA  IPMD  meetings  and 
College  of  Performance  Management  workshops.  The  col¬ 
laboration  with  the  NDIA  IPMD  Board  and  event  planners  has 
been  an  invaluable  resource  to  help  promote  new  initiatives 


Near-term  DoD  EVM  policy  updates  are  a  result  of  the  afore¬ 
mentioned  feedback  loop  and  response  to  interpretation  ques¬ 
tions  on  policy.  The  community  can  expect  in  the  next  year 
updates  to  the  following: 

■  Over  Target  Baseline/Over  Target  Schedule  Guide  ( OTB/OTS 
Guide)— the  update  will  provide  clarity  on  why  an  OTB/ 
OTS  is  needed,  what  to  do  if  it  is  needed,  and  the  process 
to  follow. 

■  DoD  EVM  System  Interpretation  Guide  (EVMSIG)— updates 
are  to  be  based  on  implementation  of  guidance  over  the 
last  year  and  a  half. 

■  Agile  and  Earned  Value  Management:  A  Program  Manager's 
Desk  Guide— updates  are  to  discuss  effect  of  Agile  on  IBRs 
and  how  Agile  metrics  should  work  with  EVM  metrics. 

■  IPMR  Implementation  Guide— potential  updates  will  move 
away  from  formats  and  toward  data-centric  delivery  of 
information. 

■  EVM  Implementation  Guide  (EVMIG)-a  revival  of  a  one- 
stop  document  for  all  EVM  application  and  guidance 
information. 

■  DFARS— updates  to  reflect  the  $100  million  EVMS 
threshold  class  deviation  memo  and  in  concert  with  up¬ 
dates  in  the  January  2017  DoDI  5000.02. 

These  policy  initiatives,  along  with  the  outreach  efforts  to  gov¬ 
ernment  and  industry,  enable  PARCA  to  ensure  the  balance 
between  oversight  and  system  impacts.  PARCA  believes  EVM 
should  be  a  self-evidently  beneficial  methodology  that  pro¬ 
vides  PMs  accurate  and  timely  insight  into  cost,  schedule  and 
technical  performance  of  DoD  programs.  Through  the  changes 
coming  to  leadership  and  organization  within  DoD,  PARCA  will 
continue  providing  succinct  guidance  on  the  interpretation  and 
implementation  of  EVM.  & 

The  authors  can  be  contacted [  or  information  sought  about  the  DoD's  EVM  policy 
through  the  PARCA  EVM  Division i,  atosd.dodevm@mail.mil. 
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Don  O'Nei 

’ll 

Let's  take  a  look  at  the  elements  of  the  Critical  Infrastructure  Resilience  Integra¬ 
tion  Engineering  process,  management,  and  engineering  considerations— with 
attention  to  the  roles  of  the  resilience  integrator  and  the  intelligent  middlemen 
and  the  convincing  evidence  provided  by  Earned  Value  Analytics  used  in  cal¬ 
culating  resilience  risk. 

The  critical  infrastructure  is  the  industrial  base  on  which  the  competitiveness  and  security  of  the  nation  depend. 
The  current  state  of  critical  infrastructure  is  at  risk  now  that  the  Internet  has  become  the  nation's  central  nervous 
system,  in  both  the  private  and  the  public  sphere.  That  critical  infrastructure  vulnerability  to  natural  disasters 
and  cascading  cybersecurity  attacks  continues.  In  fact,  software  has  become  the  critical  infrastructure  within 
the  critical  infrastructure.  It  is  here— in  the  mashup  among  an  immature  software  profession,  a  vulnerable 
cybersecurity  environment,  and  diverse  and  interdependent  industry  sectors— that  the  challenge  of  system- 
of-systems  resilience  is  born. 


O'Neill  was  president  of  the  Center  for  National  Software  Studies  from  2005  to  2008.  After  27 years  with  IBM's  Federal  Systems  Division , 
he  completed  a  3-year  residency  at  Carnegie  Mellon  University's  Software  Engineering  Institute  (SEI)  under  IBM's  Technical  Academic 
Career  Program  and  has  served  as  an  SEI  Visiting  Scientist  and  a  Litigative  Consultant  with  the  Department  of  Justice.  A  seasoned  software 
engineering  manager ;  technologist  and  independent  consultant ,  he  has  a  Bachelor  of  Science  degree  in  Mathematics  from  Dickinson  College 
in  Carlisle ,  Pennsylvania. 
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issues  of  cybersecurity  where  there  is  no 
unified  theory. 

Based  on  nearly  50  indicators  of  resilience, 
Resilience  Earned  Value  Analytics  employ 
the  most  convincing  evidence  available  to 
measure  the  degree  to  which  the  resilience 
value  proposition  is  being  achieved  both 
collectively  and  in  each  industry  sector  and 
the  degree  to  which  unattended  resilience 
risk  persists. 


Figure  1.  Vertical  Protection  and  Horizontal  Resilience 

Banking  &  Telecomm  Electrical 
Finance 


Source  of  figures  and  table:  The  author 

A  more  useful  approach  to  critical  infrastructure  resiliency 
is  needed,  one  that  represents  new  thinking  and  perhaps 
a  new  paradigm  based  on  adaptive  measures  rather  than 
simply  error  discovery,  more  selective  Internet  usage  based 
on  proven  user  protections,  credible  deterrence  through  de¬ 
monstrable  strength,  and  convincing  evidence  provided  by 
Earned  Value  Analytics. 

Contrary  to  some  assumptions,  resilience  is  not  primarily 
threatened  by  errors.  Rather,  it  primarily  is  threatened  by 
multiple  contexts  and  opposing  goals.  Successful  approaches 
to  resilience  are  tied  to  harmonizing  components  and  their 
adaptive  capability.  Nor  should  the  goal  of  resilience  simply 
be  to  bounce  back  to  the  earlier  state  before  the  moment  of 
adversity  or  attack.  It  was,  in  fact,  the  earlier  state  that  proved 
vulnerable.  Herein  lies  the  challenge. 

Cybersecurity  is  not  simply  a  problem  awaiting  a  technology 
solution.  Instead  cybersecurity  is  a  problem  of  user  behavior 
awaiting  a  deeper  awareness  and  an  acceptance  of  respon¬ 
sibility.  Simply  put,  individuals  and  organizations  that  cannot 
afford  to  lose  data  or  information  should  not  put  it  on  the  In¬ 
ternet.  Those  that  can  afford  to  lose  data  or  information  but 
would  rather  not  to  do  so  should  exercise  user  protection  by 
employing  three-factor  authentication  and  encryption. 

On  the  big  stage,  Critical  Infrastructure  resilience  is  part 
of  the  February  2017  Defense  Science  Board's  Cyber  De¬ 
terrence  Strategy  and  its  deterrence  by  denial  and  by  cost 
imposition.  A  resilient-ready  Critical  Infrastructure  would 
achieve  adversary  deterrence  by  cost  imposition  and  per¬ 
haps  even  deterrence  by  denial  of  particular  objectives  so 
long  as  there  is  a  convincing  credible  will  and  demonstrable 
capability.  Like  Game  Theory,  a  strategy  of  deterrence  is 
based  on  rationality.  Rationality  in  the  context  of  cyber¬ 
security  cannot  be  assured  when  unprepared  managers, 
politicians  and  lawyers  are  engaged  in  the  deeply  technical 


Resilience  and  Protection 

One  respected  researcher  seemed  to  con¬ 
cede  the  high  ground  of  resiliency— that  is, 
avoidance— and  associated  resilience  with 
the  old  Timex  slogan,  "Takes  a  licking  and 
keep  on  ticking."  The  question  then  be¬ 
comes:  What  perimeter  is  being  secured? 

In  protecting  a  network  node  or  a  physical  facility  in  a  geo¬ 
graphic  region,  each  node  or  facility  has  to  be  protected  and 
made  survivable  (Figure  1).  In  achieving  resilience,  propagation 
and  cascading  effects  across  the  network  and  region  must 
also  be  curtailed.  This  is  made  difficult  by  the  context  and 
culture  challenges  of  the  industry  sectors  within  the  critical 
infrastructure.  The  capabilities  needed  to  impact  crosscut¬ 
ting  issues  cannot  be  expected  to  evolve  in  a  loosely  coupled 
environment.  They  must  be  holistically  specified,  architected, 
designed,  implemented  and  tested  if  they  are  to  operate  with 
resilience  under  stress.  A  management,  process  and  engineer¬ 
ing  maturity  framework  is  necessary  to  advance  the  assurance 
of  software  security,  business  continuity,  system  survivability 
and  system-of-systems  resiliency  capabilities. 

Resiliency  Defined 

Resiliency  is  the  ability  to  anticipate,  avoid,  withstand,  mini¬ 
mize  and  recover  from  the  effects  of  adversity,  whether  natural 
or  man-made,  under  all  circumstances  of  use.  Resiliency  ap¬ 
plied  to  a  system  of  systems  focuses  on  crosscutting  issues. 
Crosscutting  effects  stem  from  dependent  relationships.  Some 
dependent  relationships  are  planned  and  intended  interac¬ 
tions  between  industry  sectors,  such  as  financial  transactions 
embedded  in  telecommunications,  electrical,  transportation, 
and  medical  operations.  Other  dependent  relationships  are 
indirect  and  stem  from  outsourced  commoditized  services 
that  bring  with  it  opportunities  for  common  single  point  fail¬ 
ures  among  industry  sectors,  such  as  the  Internet,  the  Global 
Positioning  System,  Federal  Express,  IBM,  and  Microsoft. 

In  order  to  operationalize  resiliency,  objectives  must  be 
matched  with  well-coordinated  features.  The  objective  of 
being  able  to  anticipate  calls  for  the  features  of  harmonized 
domain  engineering,  coordinated  recovery  time  objectives, 
cascade  trigger  identification,  and  digital  situation  aware¬ 
ness.  The  objective  of  avoiding  the  negative  effects  calls 
for  the  features  of  shutdown,  defense  in  depth,  operation 
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sensing  and  monitoring,  and  distributed  supervisory  control. 
The  objective  of  being  able  to  withstand  the  negative  effects 
calls  for  the  features  of  enterprise  security,  business  process 
continuity,  survivability  and  alternate  site.  And  minimizing  the 
negative  effects  calls  for  the  features  of  adaptation  manage¬ 
ment,  alternate  mode,  minimum  essential  mission  and  shut¬ 
down.  The  ability  to  recover  calls  for  the  features  of  capability 
to  reorganize,  assured  availability,  information  and  data  recov¬ 
ery,  and  cleanup  and  reconstitution. 

Context  and  Culture  Challenges 

The  critical  infrastructure  includes  the  numerous  industry  sec¬ 
tors  that  do  the  heavy  lifting.  There  are  diverse  and  complex 
operations  within  the  industry  sectors  of  the  critical  infrastruc¬ 
ture.  These  industry  sectors  comprise  an  accidental  system  of 


property,  privacy,  and  antitrust  litigation.  Strong  government 
control  facilitates  compliance  for  the  benefit  of  the  common 
good  at  the  expense  of  initiatives  taken  in  the  service  of  self- 
interest.  Industry  sector  public  expectation  and  confidence 
vary  with  respect  to  trust,  loyalty  and  satisfaction.  The  finan¬ 
cial  and  medical  sectors  depend  on  public  trust.  The  elec¬ 
trical  and  telecommunication  sectors  depend  on  customer 
loyalty  and  satisfaction.  The  diverse  industry  sector  expec¬ 
tations  of  trust,  loyalty  and  satisfaction  must  be  respected, 
blended  and  harmonized. 

Well-defined  context  and  culture  harmonization  steps  are 
needed  to  achieve  Critical  Infrastructure  Resilience.  Formal¬ 
ity  within  an  architectural  framework  facilitates  the  impo¬ 
sition  of  distributed  supervisory  control,  interoperability, 


In  operation)  these  sector  systems  may  inadvertently 

***  .  %mmm I— i— & 

impose  their  own  context  arid  culture  on  others  and  clash 
with  uncertain  and  unintended  operational  results. 


opera 


systems  that  intersect  operationally  without  a  plan  and  design 
in  advance. 

Each  sector  system  was  constructed  within  its  own  context 
and  culture.  In  operation,  these  sector  systems  may  inadver¬ 
tently  impose  their  own  context  and  culture  on  others  and 
clash  with  uncertain  and  unintended  operational  results.  In¬ 
dustry  sector  practice  varies  widely  in  their  domain  engineer¬ 
ing  approaches  resulting  in  diversity  in  architecture,  models 
and  patterns  that  include  their  representation.  Formality  within 
an  architectural  framework  facilitates  the  imposition  of  dis¬ 
tributed  supervisory  control,  interoperability,  and  operation 
sensing  and  monitoring  protocols. 

Industry  management  and  engineering  maturity  varies  widely. 
This  results  in  diverse  configuration  management,  frequency 
of  release,  conformity  to  requirements,  and  traceability  among 
life-cycle  artifacts.  Strong  code  management  practices  facili¬ 
tate  reconfiguration  and  reconstitution.  Industry  sector  prac¬ 
tice  varies  widely  in  fielding  and  operating  practices  resulting 
in  diversity  in  accountability  and  control,  supply  chain  man¬ 
agement,  civility  and  pushback,  and  willingness  to  expend  "off 
the  clock"  effort.  Exercising  strong  control  over  the  workforce 
facilitates  business  continuity  and  survivability. 

Government  regulation  imposes  varying  industry  sector 
impacts  regarding  export  control,  tax  policy,  intellectual 


and  operation  sensing  and  monitoring  protocols.  Cascading 
and  propagating  triggers  must  be  anticipated,  avoided,  and 
minimized  including  insource,  open  source,  and  outsource. 
Industry  sector  software  sourcing  exposures  must  be  under¬ 
stood  and  managed— including  insource,  open  source  and 
outsource.  Supply  Chain  Risk  Management  operations  must 
be  assured  as  must  cybersecurity  strategy  policy  decisions 
and  defined  tactics. 

Resilience  Integrator 

A  resilience  integrator  is  needed  to  organize,  integrate  and 
harmonize  industry  sectors  of  the  critical  infrastructure  into 
a  resilient  system  of  systems.  The  stakeholder  vision  for  this 
project  is  an  opportunity  value  proposition  for  operational  re¬ 
silience.  If  resilience  is  to  be  achieved,  the  resilience  integrator 
must  be  prepared  to  provide  resiliency  engineering  features 
capable  of  meeting  stringent  objectives. 

The  resilience  integrator  should  harmonize  the  context  and 
culture  of  the  numerous  industry  sectors  and  anticipate  do¬ 
main  engineering  clashes  in  order  to  avoid  unintended  opera¬ 
tions  results  stemming  from  diversity  in  management,  pro¬ 
cess,  and  engineering  approaches.  The  resilience  integrator 
should  groom  intelligent  middlemen  to  facilitate  the  adoption 
of  the  way  of  working  within  the  industry  sectors  of  the  critical 
infrastructure.  The  resilience  integrator  then  shall  facilitate  the 
resilience  maturity  of  management,  process  and  engineering 
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Figure  2.  Software  Systems  Architecture  for  Critical  Infrastructure  Resilience 


Respondent  System 


System  Assets 


capabilities  and  solutions  that  address  security,  continuity,  sur¬ 
vivability  and  resilience  among  the  industry  sector  system  of 
systems.  The  resilience  integrator  shall  specify  a  system-of- 
systems  architecture  that  facilitates  the  harmonious  coopera¬ 
tion  among  industry  sectors;  provides  digital  situation  aware¬ 
ness;  allows  for  distributed  supervisory  control  under  stress; 
and  manages  the  assembly,  delivery  and  control  of  common 
system  assets. 

The  resilience  integrator  shall  prepare  and  coordinate  a  Re¬ 
silience  Integration  Program  Plan  harmonizing,  facilitating, 
specifying,  engineering,  developing,  integrating  and  field¬ 
ing  the  critical  infrastructure  system  of  systems.  The  resil¬ 
ience  integrator  shall  frame  a  way  of  working  to  manage  the 
communication,  command,  control,  commitments  and  per¬ 
formance  between  the  industry  sectors,  their  contractors 
and  the  resilience  integrator— including  executive  councils, 
steering  groups,  working  groups  and  support  groups.  The 
resilience  integrator  shall  calculate  resilience  risk  based  on 
Earned  Value  Analytics. 

Intelligent  Middlemen 

If  the  critical  infrastructure  is  to  be  resilient,  its  sector  manag¬ 
ers  and  systems  must  respond  to  guidance  from  intelligent 
middlemen  whose  influence  is  felt  before,  during  and  after  a 
crisis.  Intelligent  middlemen  possess  the  broad  range  of  hard 
and  soft  skills  that  span  the  cultural,  ethical,  legal,  business, 
process,  management  and  engineering  dimensions  to  meet 
the  critical  infrastructure  challenges  in  anticipating,  avoiding, 
minimizing,  withstanding  and  recovering  from  crosscutting 
effects  and  to  impede  the  emergence  of  propagating  and 
cascading  effects. 

The  intelligent  middlemen  are  positioned  at  the  center  of 
things  as  the  traffic  cops  for  identifying  and  driving  resolution 
of  crosscutting  issues.  From  this  vantage  point,  the  intelligent 
middlemen  are  able  to  obtain  superior  situational  awareness. 
For  example,  they  ensure  that  recovery  time  objectives  are  co¬ 
ordinated,  interoperability  protocols  are  followed,  distributed 
supervisory  control  functions  are  coordinated,  and  operation 
sensing  and  monitoring  functions  are  applied. 


The  Maturity  Framework  for  Assuring  Resiliency  Under  Stress 
provides  the  way  to  harmonize  operations  and  deliver  value 
through  management,  process,  and  engineering  capabilities 
and  solutions  that  address  security,  continuity,  survivability 
and  resiliency.  The  system  perimeter  focus  areas  include  com¬ 
mitment  to  a  business  case,  security  in  depth,  business  con¬ 
tinuity,  and  systems  survivability.  Beyond  that,  the  essential 
focus  areas  needed  to  extend  the  system  perimeter  to  system- 
of-systems  resiliency  include  coordinated  recovery  time  objec¬ 
tives,  interoperable  information  and  data  exchange,  operation 
sensing  and  monitoring,  distributed  supervisory  control,  and 
information  and  data  recovery.  To  achieve  maturity  in  the  as¬ 
surance  of  resiliency  under  stress,  the  enterprise  must  satisfy 
the  goal-based  argument  at  each  level. 

Critical  Infrastructure  Resilience  Architecture 

The  Software  Systems  Architecture  for  Critical  Infrastructure 
Resilience  calls  for  the  following  allocation  of  control,  func¬ 
tions  and  persistent  data,  and  assets  (Figure  2).  The  situation 
system  is  populated  with  the  man-made  Critical  Infrastructure 
Sectors  and  the  intelligent  middlemen  control  node.  Each  in¬ 
dustry  sector  will  register  its  profile  in  order  to  engage  with 
the  services  of  the  respondent  system. 

The  respondent  system  is  populated  with  the  system-of- 
systems  resiliency  engineering  mission  focus  areas  intended  to 
interact  with  the  situation  system  and  its  intelligent  middleman 
in  anticipating,  avoiding,  withstanding,  mitigating,  and  recover¬ 
ing  from  the  effects  of  adversity  under  all  circumstances  of  use. 

The  system  assets  is  populated  with  the  facilities  focus  areas 
and  their  services  to  be  instantiated  by  the  respondent  sys¬ 
tem  in  its  interaction  with  the  situation  system  and  its  critical 
infrastructure  sectors. 

More  specifically: 

(1)  The  critical  infrastructure  situation  system  is  composed 
of  utilities  and  energy,  banking  and  finance,  medical  systems, 
transportation,  and  telecommunications  as  well  as  public 
health,  food  and  agriculture,  federal  agency,  regional  response, 
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Table  1.  Resilience  Risk  and  Earned  Value  Calculation 


Assurance 

Evidence 

(Weighted 

Example) 

Risk 

Indicator 

(Score 

1-5) 

Indicator 

Weight 

Electrical 

Sector 

Telecom 

Sector 

Banking 
&  Finance 
Sector 

Transport. 

Sector 

Medical 

Sector 

Weighted 

Score 

Earned 

Value 

Risk 

Sector  Weight 

5 

4 

3 

3 

2 

A.  Culture 
and  Content 
Harmonization 

164 

4 

800 

624 

468 

312 

160 

2,364 

0.6952 

0.3048 

B.  Intelligent 
Middlemen 

98 

2 

240 

292 

144 

84 

48 

708 

0.8329 

0.1671 

C.  Resiliency 
Maturity 

70 

5 

325 

260 

240 

210 

140 

1,175 

0.6911 

0.3089 

D.  System 
of  Systems 
Architecture 
Adoption 

38 

5 

200 

160 

129 

105 

70 

655 

0.5137 

0.4863 

E.  Integration 
Engineering 

57 

2 

130 

104 

72 

60 

36 

402 

0.5911 

0.4089 

F.  Way  of 
Working 

141 

3 

420 

336 

333 

234 

132 

1,455 

0.7132 

0.2868 

G.  Resilience 
Assurance  and 
Risk  Calculation 

77 

2 

170 

136 

102 

78 

52 

538 

0.5274 

0.4726 

H.  Cyber¬ 
security 

Strategy  and 
Tactics 

78 

3 

240 

192 

162 

126 

84 

804 

0.5254 

0.4746 

Total  Indicators 

723 

2,525 

2,104 

1,641 

1,209 

722 

8,101 

0.6483 

0.3517 

emergency  response,  and  intelligent  middlemen  control  node. 
Each  sector  of  the  Situation  System  is  composed  of  control, 
functions  and  persistent  data,  and  assets  local  to  the  sector. 

(2)  The  respondent  system  is  engineered  as  a  system-of- 
systems  architecture  composed  of  coordinated  recovery  time 
objectives,  interoperable  information  and  data  exchange,  op¬ 
eration  sensing  and  monitoring,  digital  situation  awareness, 
distributed  supervisory  control,  and  information  and  data  re¬ 
covery.  The  respondent  system  is  composed  of  the  control, 
functions  and  persistent  data,  and  assets  of  the  global  system 
of  systems. 

(3)  The  system  assets  include  services  and  facilities  to  be  in¬ 
stantiated  by  sector:  Enterprise  Security  Commitment  Man¬ 
agement  Services,  Enterprise  Business  Process  Continuity 
Services,  and  System  Survivability  Engineering  Services.  The 
system  assets  are  composed  of  the  facilities  and  service  assets 
made  available  to  the  situation  system  and  its  sectors  by  the 
respondent  system. 


Risk  Calculation  Worksheet 

The  Resilience  Risk  and  Earned  Value  Calculation  worksheet 
(Table  1)  is  intended  to  provide  a  useful  point  of  reference. 
The  worksheet  is  used  to  compile  the  assurance  evidence 
element  scores  of  risk  indicators  for  each  of  five  industry 
sectors.  Each  indicator  is  scored  1  (low)  to  5  (high).  Both 
the  assurance  evidence  elements  and  the  industry  sectors 
are  assigned  weights  of  1  to  5.  This  Critical  Infrastructure 
Resilience  dashboard  is  intended  to  shine  a  spotlight  on 
resilience  risk  and  earned  value  in  order  to  reveal  gaps,  sug¬ 
gest  vulnerabilities  and  point  the  way  forward  for  participat¬ 
ing  industry  sectors. 

The  Way  Forward 

So  there  we  have  the  user  story  for  Critical  Infrastructure 
Resiliency.  Resiliency  is  the  High  Ground...  claim  it!  Make 
your  move!  & 


The  author  can  be  contacted  at  oneilldon@aol.com. 
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he  answer  to  tackling  any  large  federal  acquisition  resides 
in  the  old  adage  of  "How  do  you  eat  an  elephant?  One  bite 
at  a  time."  We  ignore  the  advice  "one  bite  at  a  time"  and 
in  doing  so,  are  consumed  by  the  Department  of  Defense 
(DoD)  acquisition  process. 


Tackling  an  Acquisition  Category  (ACAT)  I  program  with  its  cornucopia  of  required  acquisition  documentation 
and  full  menu  of  processes  and  reviews  in  one  bite  seems  insurmountable  at  best  and,  at  worst,  impossible  to 
achieve.  However,  if  you  view  "eating  the  elephant"  as  an  opportunity  to  modify  and  streamline  the  acquisition 
process  in  order  to  obtain  more  comprehensive  results  (one  bite  at  a  time),  then  the  "elephant"  becomes  much 
easier  to  consume. 


If  you  are  fortunate  enough  to  stand  up  an  acquisition  program,  you  have  a  unique  opportunity  that  expands 
both  knowledge  and  experience.  Removing  the  burden  of  legacy  decisions  provides  the  perfect  opportunity 
to  mold  the  program  from  your  perspective.  This  article  discusses  the  tailored  acquisition  approach  of  a 


Bernhard,  an  acquisitions  analyst  for  the  Navy  Enterprise  Business  Solutions  (EBS)  Program  Management  Office  (PMO),  is  a  former  Navy 
Supply  Corps  Officer.  He  served  in  various  assignments  as  Supply  Officer  Afloat  and  as  Staff  N4  Contracting  and  Logistics  Officer  for  U.S. 
Naval  Forces  Alaska  and  Coast  Guard  District  17.  He  has  been  an  acquisitions  analyst  at  several  DoD  Program  Offices.  Caffrey,  a  Navy 
EBS  PMO  acquisition  manager ;  is  a  former  naval  officer  and  program  manager  for  the  Navy's  overseas  network  (ONE-NET)  that  serves  the 
business  and  operational  needs  of  133,000  Navy  users  in  Europe  and  the  Middle  East  and  Far  East.  She  previously  was  Commanding  Of¬ 
ficer,  U.S.  European  Command  Navy  Reserve  Detachment  130  in  Stuttgart,  Germany,  and  vice  president  for  a  defense  firm  supporting  Army 
acquisition  management  in  Washington  D.C.  She  has  worked  with  the  PEO  Enterprise  Information  Systems  since  2007. 
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It  is  important  to  note  that  any  program  seeking  to  tailor 
programmatic  requirements  must  secure  approval  from  a  Milestone 

Decision  Authority  (JVIDA)  via  an  Acquisition  Decision  Memorandum 

■*>  * 

prior  to  implementing  any  tailoring  methods. 

*  * 

*  -  m 


commercial-off-the-shelf  (COTS)  solution  for  the  Depart¬ 
ment  of  the  Navy  (DoN)  Electronic  Procurement  System 
(ePS),  a  Major  Automated  Information  System  and  an  ACAT 
IAC  Defense  Business  System  (DBS). 

Ask  anyone  to  describe  the  DoD  Acquisition  Program  Life 
Cycle  and  the  response  probably  will  reference  the  DoD  In¬ 
struction  rather  than  provide  a  succinct,  descriptive  answer. 
Instead  of  a  deep  dive  into  acquisition  regulations,  let's  look 
at  the  beginning  phases  of  the  acquisition  life  cycle  and  high¬ 
light  the  DBS  management  activities  accomplished  by  Navy 
Enterprise  Business  Solutions  (PMW  220). 

The  point  where  a  program  officially  begins  depends  on 
how  much  preliminary  work  has  been  accomplished  on  it. 
Ideally,  a  DBS  program  begins  strategizing  about  acquisi¬ 
tion  tailoring  early  in  the  Materiel  Solution  Analysis  (MSA) 
phase.  It  is  accepted  business  practice  to  tailor  all  acquisi¬ 
tion  programs  wherever  possible  to  improve  cost,  schedule 
and  performance. 

Navy  ePS  grew  out  of  a  need  to  revitalize  or  replace  the 
existing  DoD  Contract  Writing  System  (CWS),  known  as 
the  Standard  Procurement  System  (SPS),  used  by  every 
Service.  The  DoD  developed  and  deployed  SPS  in  the  late 
1990s.  With  increases  in  program  and  contract  sizes,  the 
DoN  needed  a  new,  fully  integrated  and  auditable  CWS. 
The  MSA  phase  identified  an  estimated  723  distinct 
requirements.  Market  research  confirmed  that  several 
COTS  solutions  could  fit  the  bill.  An  analysis  by  the  ePS 
team  determined  that  a  COTS  solution  could  eliminate  a 
large  portion  of  the  Technology  Maturation  phase.  This 
set  the  stage  for  the  ePS  program  to  embrace  a  proactive 
tailored  approach  as  an  official  part  of  its  Acquisition 
Strategy.  It  is  important  to  note  that  any  program  seeking 
to  tailor  programmatic  requirements  must  secure  ap¬ 
proval  from  a  Milestone  Decision  Authority  (MDA)  via 
an  Acquisition  Decision  Memorandum  prior  to  imple¬ 
menting  any  tailoring  methods.  With  this  in  mind,  the 
ePS  team  set  out  to  identify  the  regulatory  and/or  statu¬ 
tory  documents  that  would  be  candidates  for  a  waiver 
or  tailoring.  The  team  researched  tailoring  methods  and 
decided  to  implement  a  dynamically  tailored  acquisition 
approach  that  included  a  modified  Acquisition  Model  3, 


combined  technical  reviews,  and  integrated  the  Devel¬ 
opmental  and  Operational  Test  phases. 

Documentation  tailoring  began  with  regulatory  documenta¬ 
tion  requirements  for  the  Request  for  Proposal-Release  Deci¬ 
sion  (RFP-RD)  point.  The  most  significant  tailored  requirement 
was  combining  the  Acquisition  Strategy  and  Acquisition  Plan 
(AS/AP)  into  one  document.  The  combination  of  these  very 
similar  documents,  rather  than  staffing  two  separate  acquisi¬ 
tion  documents,  saved  time  and  manpower. 

The  Capability  Development  Document  and  Capability  Pro¬ 
duction  Document  were  replaced  by  the  Capability  Require¬ 
ments  Document  (CRD).  Signed  on  Feb.  4,  2016,  the  CRD 
was  a  significant  accomplishment  for  the  ePS  team.  CRD  ap¬ 
proval  means  that  the  program  can  use  System  Performance 
Measures  (SPMs)  instead  of  the  Joint  Capabilities  Integra¬ 
tion  and  Development  System  Key  Performance  Parameters. 
SPMs  capture  the  minimum  operational  effectiveness  and 
suitability  attributes  required  to  achieve  the  overall  desired 
capabilities  for  ePS.  SPMs  also  contain  threshold  and  objective 
values  verifiable  by  testing  and  other  analysis.  Since  ePS  will 
be  DoN-specific,  joint  requirements  are  not  necessary.  This 
also  meant  ePS  would  not  be  a  Joint  Requirements  Oversight 
Council  (JROC)  interest  program  or  subject  to  J ROC  oversight. 

As  the  ePS  program  continued  to  proceed,  other  documen¬ 
tation  fell  into  place.  A  rough  order  of  magnitude  (ROM) 
cost  estimate  was  developed  in  October  2016  in  lieu  of  an 
independent  cost  estimate.  The  Cost  Analysis  Require¬ 
ments  Document,  Life  Cycle  Sustainment  Plan,  Program 
Protection  Plan,  which  includes  the  Cybersecurity  Strategy 
and  System  Engineering  Plan,  were  all  approved  at  the  Pro¬ 
gram  Manager  or  Program  Executive  Office  level  by  Dec. 
5,2016. 

The  ePS  team  tailored  the  acquisition  approach  in  coordination 
with  the  Service  Acquisition  Executive  and  the  Office  of  the 
Secretary  of  Defense  (OSD),  with  approval  from  the  MDA.  It 
is  important  to  note,  that  Frank  Kendall,  then  the  Under  Sec¬ 
retary  of  Defense  for  Acquisition,  Technology,  and  Logistics 
(USD[AT&LJ),  stated  that,  "tailoring  should  and  could  be  dis¬ 
cussed  in  the  Acquisition  Strategy.”  The  comment  granted 
ePS  the  flexibility  to  pursue  innovative  tailoring  methods  as 
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Origin  of  the  Navy's  ePS 


The  idea  for  a  Navy  ePS  grew  out  of  a  need  to  revitalize  or 
replace  CWS,  known  as  the  Standard  Procurement  System 
(SPS),  developed  and  deployed  by  the  DoD  in  the  1990s  for 
the  military.  SPS  has  been  the  DoN's  primary  contract  writing 
system  and  contracting  data  exchanger.  Initially  developed  for 
simplified  acquisitions,  today  SPS  issues  more  than  75  percent 
of  DoN  contracts  and  modifications. 

Patched  and  updated  several  times  to  increase  its  longevity 
and  usefulness,  SPS  was  never  intended  for  complex  major 
contracts.  The  need  for  revitalization  or  an  entirely  new  sys¬ 
tem  was  quickly  becoming  more  apparent.  The  Navy  needed 
a  new  contract  writing  capability,  able  to  meet  the  contract 
writing  requirements  of  the  21st  century  for  effective  deal 
initiations,  management,  termination,  close  out,  and  audit- 
ability  across  the  full  spectrum  of  Navy  and  Marine  Corps 
contracting  environments. 

Recognizing  the  need  for  a  new  or  updated  CWS,  the 
USD(AT&L)  initially  directed  the  development  of  a  new  CWS 
and  a  halt  in  the  use  of  SPS  by  Sept.  30,  2017.  That  date  has 
been  revised  to  no  new  awards,  agreements,  or  orders  by  Sept. 
30, 2020,  with  the  system  to  "sunset"  by  Sept.  30, 2023.  Mili¬ 
tary  departments  and  agencies  were  delegated  the  respon¬ 
sibility  to  fund,  develop  and  field  a  replacement  CWS  within 
the  DoD  Business  Enterprise  Architecture  (BEA)  boundaries 
while  ensuring  data  standardization,  electronic  data  transfer, 
and  data  visibility  across  all  relevant  systems. 

In  addition  to  its  use  as  a  CWS,  Navy  ePS  will  also  serve  as  the 
DoN's  system  of  record  for  the  official  contract  file.  The  system 
will  have  the  flexibility  to  interface  with  Navy,  the  DoD  and 
federal  electronic  systems  as  needed  to  meet  all  stakeholder 
requirements.  Data  captured  through  system  automation  and 
distributed  through  interface  transactions  are  critical  elements 
in  supporting  the  DoN's  Financial  Improvement  Program  and 
enabling  the  DoN's  vision  of  clean  financial  statements.  The 
ePS  also  will  comply  with  the  National  Defense  Authorization 
Act  of  2013  direction  that  DoD  writes  all  contracts  electroni¬ 
cally  by  2018. 

The  Program  began  at  the  MSA  phase  with  the  Analysis  of 
Alternatives  (AoA)  Study  Plan  in  September  2012,  followed 


deemed  appropriate,  yet  required  the  full  documentation  and 
justification  of  those  methods  in  the  AS/AP,  for  review  and 
approval  by  the  MDA. 

With  Kendall's  support  in  hand,  the  tailoring  approach 
was  set  and  Navy  ePS  marched  toward  issuing  a  RFP.  As 
a  result  of  the  early  groundwork,  program  events  were 
accomplished  in  quick,  rapid-fire  succession.  Gate  3  was 
completed  on  May  3,  2016.  Gate  4  was  completed  June  1, 
2016.  Gate  5  was  completed  Dec.  20,  2016.  The  effective 
tailoring  by  the  ePS  team  resulted  in  a  favorable  RFP-RD 
by  the  MDA  on  Jan.  18,  2017,  and  signature  approval  of  the 
combined  AS/AP  on  Jan.  24,  2017.  Also,  James  A.  Mac- 
Stravic,  performing  the  duties  of  the  USD(AT&L),  reclassi¬ 
fied  ePS  as  an  ACAT  IAC  acquisition  program  and  delegated 


a  few  months  later  by  the  AoA  in  2013.  Navy  ePS  achieved 
a  Materiel  Development  Decision  on  April  1,  2013.  The  Navy 
Resources  and  Requirements  Review  Board  approved  the  ePS 
AoA  in  May  2014.  The  preferred  alternative  in  the  AoA  is  a 
COTS  CWS  with  COTS  middleware  that  provides  an  interfacing 
capability  for  the  CWS,  allowing  integration  and  maintenance  in 
the  DoN  enterprise.  Following  the  AoA's  recommendation,  the 
program  intends  to  invest  in  a  COTS  solution,  supplemented  by 
gap-closures  methodologies  such  as  Business  Process  Man¬ 
agement  tools  and  COTS  middleware. 

The  ePS  will  provide  the  DoN  with  standardized,  compre¬ 
hensive  end-to-end  (E2E)  contract  management  for  ser¬ 
vices,  supplies  and  construction.  The  ePS  also  will  support 
mission-related  assistance  using  assistance  agreements, 
such  as  grants,  and  will  deliver  the  following  functional/ 
technical  capabilities: 

■  Provide  robust  functional  capabilities  to  support  the  E2E 
Procure-to-Pay  process. 

•  Capture  data  at  the  source  (i.e.,  no  redundant  data  entry 
via  touch  labor). 

■  Leverage  a  data-centric  and  standard-compliant  design  for 
data  storage  and  exchanges. 

■  Enforce  business  process  controls  and  system  controls  to 
support  an  unqualified  audit  opinion. 

■  Inform  decision  making  through  available,  accurate,  timely 
and  visible  reporting. 

■  Comply  with  DoN-specific  architecture  and  environmental 
constraints  (e.g.,  Navy  Marine  Corps  Intranet). 

■  Deliver  a  flexible  solution  that  allows  prompt  and  cost-ef¬ 
fective  changes. 

•  Support  100  percent  of  the  DoN  contracting  actions  in  a  fully 
auditable  and  traceable  environment. 

■  Provide  a  secure  computing  environment  that  implements 
critical  security  controls  and  capabilities. 

The  result  of  a  successful  Navy  ePS  implementation  will  be  a 
contracting  process  workforce  that  is  well  informed  and  com¬ 
pletely  empowered  to  write  accurate  and  timely  contracts  in 
support  of  the  warfighter,  including  management  of  the  data 
flows  into  and  out  of  the  deal-making  space. 


MDA  responsibility  to  the  Secretary  of  the  Navy  on  March 
20,  2017.  Navy  ePS  now  stands  ready  to  release  the  RFP  to 
industry  sometime  between  late  in  the  second  quarter  and 
early  in  the  third  quarter  of  Fiscal  Year  2017. 

Navy  ePS  follows  the  Acquisition  Model  3  for  incrementally 
deployed  software  intensive  programs.  However,  since  ePS 
was  planning  on  a  COTS  solution,  the  ePS  team  streamlined 
Model  3  to  use  only  one  increment  with  several  releases,  in¬ 
stead  of  the  multiple  increments  and  subsequent  builds  that 
are  more  commonly  performed.  The  single  increment  would 
contain  a  Limited  Deployment  (LD),  a  Release  1,  a  Release  2, 
and  a  Full  Deployment  (FD).  The  ePS  team  also  concluded 
that  an  Authority  to  Proceed  could  replace  the  Milestone  B 
Decision  review. 
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The  ePS  team  learned  myriad  lessons  during  this  stage  of  the 


development  and  execution  of  its  tailoring  approach— and 

captured, them  in  a  database  for  future  activities  and  socialization 

-•  * 

among  other  program  teams. 

4  n 


To  keep  all  parties  abreast  of  evolving  tailoring  activities, 
the  ePS  team  held  recurring  Acquisition  and  Engineering 
Working  Integrated  Product  Team  meetings  with  DoN- 
OSD  stakeholders.  This  mitigated  program  risk  and  allowed 
stakeholders  swift  access  to  a  COTS  solution,  resulting  in 
accelerating  the  projected  program  schedule.  Working  ses¬ 
sions  occurred  as  part  of  the  documentation  development 
process  to  address  significant  issues  early  in  the  process 
and  to  get  real-time  stakeholder  buy-in.  This  approach 
enabled  timely  resolution  and  response  to  significant  and 
critical  issues  as  they  arose. 

The  ePS  team  learned  myriad  lessons  during  this  stage  of  the 
development  and  execution  of  its  tailoring  approach— and 
captured  them  in  a  database  for  future  activities  and  social¬ 
ization  among  other  program  teams. 

Program  Offices  engaging  in  a  DBS  should  apply  the  following 
lessons  early  as  activities  may  take  nearly  12  months  to  com¬ 
plete,  and  documentation  requirements  may  change  during 
the  development  and  staffing  process.  The  three  key  lessons 


learned  are  (1)  allow  sufficient  time  to  staff  and  process  acqui¬ 
sition  documents;  (2)  develop  standard  templates,  illustrations 
and  identify  responsible  parties;  (3)  and  collaborate  across 
functional  areas  to  ensure  production  of  greater  descriptive 
depth  and  well-written  requirements  and  documents. 

The  ePS  program  is  poised  to  deliver  an  automated  DoN 
CWS  that  adheres  to  DoD  Information  Technology  business 
principles  and  fulfills  congressionally  mandated  auditability 
and  Financial  Improvement  and  Audit  Readiness  (FIAR)  re¬ 
quirements.  In  conclusion,  the  Navy  ePS  tailored  acquisition 
approach  is  proactive,  innovative,  and  fully  supports  DoD 
acquisition  instructions  and  DoD  Better  Buying  Power  3.0 
tenets.  The  approach  encourages  thoughtful  planning  and 
early  program  assessment  to  minimize  unnecessary  bureau¬ 
cracy  and  low-value  added  processes  in  major  investment 
decisions  supporting  delivery  of  vital  operational  capability 
to  the  warfighter.  & 


The  authors  can  be  contacted  at  Jeffrey.bernhard@navy.mil  and  marykay. 
caffreyanavy.mil 
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The  Quadrennial  Defense  Review 
—and  Cyberspace 

Jennifer  Miller,  D.B.A. 


The  latest  Quadrennial  Defense  Review  (QDR)  focused  on  cyberspace  and 
the  projected  approach  of  the  Department  of  Defense  (DoD)  to  preparing 
and  performing  in  cyberspace.  This  article  considers  cyberspace  relative 
to  possible,  major  implications  for  airpower  in  future  contingency  opera¬ 
tions,  including  a  shift  in  focus  on  the  U.S.  Air  Force's  five  core  missions, 
an  increased  emphasis  on  cyberspace  warfare,  and  a  consistently  high  demand  for 
cyber  talent. 

Most  higher-ranking  DoD  officials  know  about  the  key  documents  in  the  planning  phase  of  the  Planning,  Pro¬ 
gramming,  Budgeting,  and  Execution  (PPBE)  process,  which  is  taught  as  part  of  the  DoD  Decision  Support 
System.  The  system  also  includes  the  Defense  Acquisition  System  (DAS)  and  the  Joint  Capabilities  Integration 


Miller  is  a  deputy  branch  chief  and  cost  analyst  of  the  National  Guard  Bureau  Headquarters'  Joint  Staff  in  Virginia ,  where  she  serves  both 
the  Army  National  Guard  and  Air  National  Guard.  She  earlier  supported  the  Air  Force  and  Army  at  installations  along  the  East  Coast.  She 
is  a  Certified  Government  Financial  Manager  and  a  Certified  Defense  Financial  Manager  with  an  acquisition  specialty.  Miller  received  her 
doctorate  of  business  administration  from  Walden  University's  College  of  Management  and  Technology. 
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and  Development  System  (JCIDS).  The  QDR,  a  key  PPBE 
document,  provides  a  strategic  framework  of  future  plans 
for  programmers,  budgeteers  and  the  personnel  executing 
the  PPBE  process  phases. 

As  I  completed  the  U.S.  Air  Force's  non-master's  degree  pro¬ 
gram  of  the  Air  Command  and  Staff  College,  I  had  opportuni¬ 
ties  to  reflect  on  the  reviews  and  strategic  projections  of  the 
defense  community.  One  pivotal  document  incorporated  in 
an  essay  assignment  related  the  QDR  to  cyberspace.  Those 
working  with  cyber  probably  realize  it  is  a  constrained,  costly 
and  complex  area— from  the  technology  assets  to  the  people 
involved.  With  airpower  and  the  future  in  mind,  I  considered 
major  and  high  cost  implications  that  await  those  involved. 

The  2014  QDR,  produced  in  compliance  with  U.S.  Code,  Title 
10,  Sec.  118  (a),  reviewed  the  DoD  strategy  and  priorities  to 
provide  that  "modern  warfare  is  evolving  rapidly,  leading  to 
increasingly  contested  battlespace  in  the  air,  sea,  and  space 
domains— as  well  as  cyberspace— in  which  our  forces  en¬ 
joyed  dominance  in  our  most  recent  conflicts."  Given  these 
assertions  of  modern  warfare,  I  believe  the  major  implications 
for  airpower  in  future  contingency  operations  include  a  shift 
in  focus  on  the  Air  Force's  five  core  missions,  an  increased 
emphasis  on  cyberspace  warfare,  and  a  consistently  high  de¬ 
mand  for  cyber  talent.  This  may  mean  a  variety  of  micro-  and 
macroeconomic  approaches  including  more  sign-on  bonuses, 
retention  pay,  and  other  initiatives  taken  to  attract  and  retain 
the  special  skillsets  of  the  critical  and  constrained  cyber  work¬ 
force  in  conjunction  with  yet-to-be  imagined  technological  so¬ 
lutions.  My  beliefs  in  these  major  implications  for  airpower  in 
future  contingency  operations  are  supported  and  defended 
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with  observations  made  by  glancing  to  the  past  and  grasping 
the  present  circumstance. 

The  Air  Force's  current  core  missions  stemmed  from  those 
founded  in  1947,  but  a  shift  in  content  and  application  demon¬ 
strates  the  evolution  I  project  for  the  future.  That  evolution  will 
entail  a  shift  in  focus  among  the  five  Air  Force  core  missions 
with  paramount  focus  on  the  core  missions  of  air  and  space 
superiority— and  on  intelligence,  surveillance  and  reconnais¬ 
sance  (ISR).  I  believe  these  two  core  missions  will  be  two  of 
greater  or  equal  focus  compared  to  command  and  control  (C2) 
with  the  final  two  core  missions  of  rapid  global  mobility  and 
global  strike  of  the  least  focus.  My  projections  of  this  shift 
in  focus  as  an  implication  for  airpower  in  future  contingency 
operations  are  inspired  by  authors  such  as  Dr.  Benjamin  S. 
Lambeth  (see  "Airpower,  Spacepower,  and  Cyberpower,''  in 
the  January  2011  issue  of  the  Joint  Force  Quarterly )  who  has 
described  the  battlespace  dominance  challenges  cyberspace 
invites  for  established  forces  with  decades  of  dominance  and 
irregular  warfare  actors  with  limited  histories  who  conduct  op¬ 
erations  using  rudimentary  means.  From  a  PPBE  perspective, 
I  see  some  major  shifts  and  conflicts  between  what  has  been 
set  in  place  in  the  programming  phase  of  the  PPBE  process  and 
what  will  be  budgeted  and  executed  by  directives  from  a  new 
administration  rapidly  approaching  the  production  of  its  own 
QDR.  This  shift  should  produce  a  surplus  in  some  programs, 
perhaps  enough  to  offset  the  areas  of  greater  focus  that  await 
programmer  adjustment. 

The  cyberspace  domain  also  seems  limitless  given  the  man¬ 
made,  malleable  terrain.  Cost  analysts  and  estimators  won't 
assess  transportation  costs  or  a  great  many  new  weapons 
because  cyberspace  is  more  of  an  idea  than  a  fungible  good 
or  readily  categorized  service.  From  the  domains  most  fre¬ 
quented  and  readily  recognized  in  the  Air  Force's  core  mis¬ 
sions,  I  view  cyberspace  as  the  most  rapidly  evolving  of  mod¬ 
ern  warfare  domains.  Hence,  the  contested  battlespace  stakes 
are  highest  while  the  entry  barriers  are  the  lowest.  As  a  result, 
this  domain  presumably  receives  the  greatest  focus  in  both 
contingent  and  localized  operations.  These  vast  comparisons 
of  highs  and  lows  were  described  by  Eric  Sterner  in  his  article 
on  "Retaliatory  Deterrence  in  Cyberspace,"  in  the  March  1, 
2011,  issue  of  the  George  C.  Marshall  Institute's  Science  for 
Better  Public  Policy. 

A  second  major  implication  for  future  airpower  contingency 
operations  is  the  increased  emphasis  on  cyberspace  warfare. 
Concerns  about  cyber  warfare  resemble  those  surrounding 
nuclear  threats  and  attacks  in  prior  decades;  however,  the  idea 
of  mutually  assured  destruction  that  deterred  nuclear  warfare 
does  not  possess  as  much  weight  in  cyber  war.  The  2014  QDR 
outlined  characteristics  of  the  Joint  Force  and  current  domi¬ 
nance  in  cyberspace  but  explicitly  mentioned  sustainment  pri¬ 
orities  in  science  and  technology  along  with  steps  to  continue 
progress  in  combating  "full-spectrum  cyberspace  capabilities" 
to  counter  increasing  challenges  to  continued  U.S.  dominance. 
In  my  opinion,  this  full-spectrum  of  cyberspace  includes  the 
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individuals,  hardware,  software,  networks,  technologies  and 
asymmetrical  activities.  As  you  can  imagine,  the  unknown 
equates  to  a  great  many  risks,  and  cost  estimates  will  reflect 
those  risks  in  terms  of  higher  costs.  Furthermore,  cyberspace 
is  a  global  domain  in  an  information  environment  with  inter¬ 
dependent  networks  in  a  terrain  unlike  those  of  ground,  sea 
and  air  operations  in  past  conflicts. 

Cyberspace  also  extends  to  the  private  sector  and  among 
global  partnerships  and  lacks  geographic  boundaries.  Ef¬ 
forts  to  deal  with  cyberspace  concerns  also  have  included 
"innovative  approaches  to  how  we  fight,  how  we  posture  our 
force,  and  how  we  leverage  our  asymmetric  strengths  and 
technological  advantages"  as  part  of  addressing  the  complex 
cyberspace  warfighting  environment  outlined  in  the  QDR. 
The  repeated  use  of  "how"  in  our  cyberspace  concerns  and 


advancements"  and  public  sector  entities  seem  increasingly 
reliant  on  commercial-off-the-shelf  (COTS)  technologies  that 
risk  the  vulnerabilities  recorded  in  Annex  3-12,  Cyberspace 
Operations.  The  value  of  private  industry  personnel  is  dem¬ 
onstrated  by  the  lack  of  career-broadening  possibilities  for 
DoD  personnel  in  cyber  career  fields:  There  are  significant 
education  requirements,  and  experience  would  be  difficult 
to  attain  for  a  guaranteed  return  on  investment. 

In  addition,  the  man-made  nature  of  the  cyber  domain  con¬ 
tributes  to  the  dynamic  quality  and  uniqueness  of  the  un¬ 
usual  terrain  that  must  be  navigated  and  mastered.  The  im¬ 
plication  for  airpower  in  future  contingencies  means  there 
is  a  need  for  highly  and  continually  trained  and  competent 
personnel  who  have  a  enduring  and  passionate  commit¬ 
ment  to  the  work.  This  means,  in  turn,  that  there  must  be 


The  seemingly  lawless  cyber  environment  conflicts  with  legal 
guidance  such  as  the  law  of  armed  jeonflTct,  as  well  as  the 
morals,  beliefs  and  ethics  that  a  more  junior  cyber  force  may 
struggle  to  successfully  navigate. 
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approaches  to  addressing  cyberspace  complexity  illustrates 
the  abundant  options  in  the  unique  cyberspace  war  environ¬ 
ment.  The  increased  emphasis  on  cyberspace  warfare  also 
is  evident  in  human  resource  and  personnel  matters.  Again, 
money  is  only  part  of  the  equation  for  building  and  sustaining 
the  cyber-capable  workforce. 

A  final  major  implication  for  future  contingency  air  opera¬ 
tions  is  the  anticipated,  consistently  high  demand  for  cyber 
talent.  It  can  be  presumed  the  Air  Force  will  maintain  the  lead 
in  the  cyber  realm  and  cyber  warfare,  especially  in  contingen¬ 
cies,  given  the  Air  Force  core  mission  of  air  and  space  superi¬ 
ority.  This  lead  position  automatically  will  impose  on  the  Air 
Force  the  greatest  burden  of  maintaining  a  cyber  force.  The 
2014  QDR  openly  described  how  the  DoD  will  invest  in  ex¬ 
pansion  of  cyber  capabilities  including  recruiting,  training  and 
retaining  cyber  personnel  with  specific  mention  of  the  DoD 
Cyber  Mission  Force,  for  which  manning  began  in  2016.  A 
potential  example  was  reported  by  Steve  Walsh  on  National 
Public  Radio  on  Aug.  4, 2015.  Walsh  cited  the  National  Guard 
cyber  units  where  guardsmen  maintain  a  private  sector,  full- 
benefit  position,  while  serving  their  country  with  those  same 
top-notch  skills  in  a  national  defense  capacity.  The  value  of 
private  industry  personnel  and  expertise  to  airpower  contin¬ 
gency  operations  cannot  be  overstated  as  private  industry 
is  said  to  constitute  "the  primary  catalyst  for  technological 


succession  planning  for  on-the-job  and  continuous  training 
in  a  rapidly  changing  environment.  As  described  by  retired 
Air  Force  Maj.  Gen.  Charles  J.  Dunlap  in  his  article  "Per¬ 
spectives  for  Cyber  Strategists  on  Law  for  Cyberwar,"  in 
the  spring  2011  issue  of  the  Strategic  Studies  Quarterly ,  the 
seemingly  lawless  cyber  environment  conflicts  with  legal 
guidance  such  as  the  law  of  armed  conflict,  as  well  as  the 
morals,  beliefs  and  ethics  that  a  more  junior  cyber  force 
may  struggle  to  successfully  navigate.  As  budget  and  per¬ 
sonnel  cuts  persist,  the  struggle  also  continues. 

Thus,  assertions  of  the  2014  QDR  highlight  the  rapid  evolution 
of  modern  warfare  to  include  cyberspace.  I  hypothesize  that 
the  major  implications  for  future  airpower  contingency  opera¬ 
tions  include  a  shift  in  focus  on  the  Air  Force's  five  core  mis¬ 
sions  and  potentially  the  other  military  Services'  mission  sets. 
This  shift  in  core  mission  focus  correlates  with  an  increased 
emphasis  on  cyberspace  warfare,  and  melds  with  a  consis¬ 
tently  high  demand  for  cyber  talent.  The  costs  of  the  shifts 
and  cyberspace  focus  will  be  substantial,  but  the  planning  has 
started.  Now  the  burden  rests  on  the  programmers,  budge- 
teers  and  executing  parties.  My  beliefs  about  these  major  were 
inspired  by  my  studies  for  Air  Command  and  Staff  College  as 
well  as  observations  of  current  trends.  & 


The  author  can  be  contacted  at  jammrellim@yahoo.com. 
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An  Author-Historian  Takes  a  Fresh  Look 


A  small  and  relatively  isolated  country,  distant  from  the 
centers  of  world  power,  struggles  to  establish  inde¬ 
pendence  and  self-government— despite  a  critical 
shortage  of  up-to-date  weapons  and  materiel.  A  great 
military  power  steps  in  and  provides  badly  needed 
training  and  war  supplies,  and  even  troops  and  naval  support,  so 
that  victory  is  no  longer  an  elusive  goal  but  a  probable  outcome. 

Are  we  describing  United  States  intervention  in  the  Middle  East?  Or  Asia?  No.  We  are 
looking  through  the  lens  of  history  at  our  own  nascence  as  an  independent  nation  among 
nations  and  the  story  of  French  and  Spanish  support  for  our  cause. 

Larrie  D.  Ferreira's  recent  book,  "Brothers  at  Arms:  American  Independence  and  the 
Men  of  France  and  Spain  Who  Saved  It,"  examines  the  role  of  the  French,  aided  and 
abetted  by  the  Spanish,  in  securing  this  country's  independence  by  their  intervention  in 
our  Revolutionary  War  and  expansion  of  that  conflict  into  other  theaters.  Our  European 


Above:  The  book's  au¬ 
thor,  Larrie  D.  Ferreiro 
Below:  The  book 
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allies  supplied  critically  needed  weapons  (muskets  and  can¬ 
nons)  as  well  as  ammunition,  gunpowder,  uniforms,  and  the 
naval  forces  that  proved  decisive  in  bottling  up  Lord  Cornwallis' 
troops  at  Yorktown,  Virginia— leading  to  the  British  surrender 
there.  The  464-page  book  ($30  per  copy)  was  published  by 
Alfred  A.  Knopf,  New  York,  in  November  2016. 

At  the  time  of  our  War  for  Independence,  the  British  Colonies 
of  North  America  primarily  were  agrarian  societies,  and  their 
largest  exports  (usually  through  Britain,  as  required  by  law) 
consisted  of  raw  resources:  cotton,  tobacco,  hemp,  furs,  rice, 
indigo,  dried  fish,  and  a  circular  trade  of  rum  and  molasses 
involved  in  the  Caribbean  sugar  islands'  slave  trade. 

Most  Americans  are  taught  as  schoolchildren  of  poorly  outfit¬ 
ted  Continental  soldiers,  armed  perhaps  with  hunting  weapons 
and  fighting  with  their  feet  wrapped  in  rags,  the  winter  at  Val¬ 
ley  Forge,  and  Washington  crossing  the  Delaware  to  surprise 
Britain's  Hessian  troops  and  win  victory  at  Trenton,  New  Jer¬ 
sey.  But  while  it  also  is  learned  that  European  officers  such  as 
the  French  Generals  Rochambeau  and  Lafayette,  the  Polish 
Generals  Kosciuszko  and  Pulaski,  and  the  Germans  de  Kalb 
and  von  Steuben  afforded  great  help  to  the  American  cause, 
little  usually  is  said  about  the  general  and  dangerous  short¬ 
age  of  weapons  and  ammunition  or  how  the  Revolution  would 
likely  have  failed  without  material  help  from  foreign  nations. 
This  was  a  critical  problem  despite  the  thousands  of  miles  of 
ocean  separating  the  Americans  from  their  Colonial  master, 
the  preeminent  naval  power  of  Europe. 

There  were  skilled  gunsmiths  in  the  Colonies,  but  each  could 
produce  perhaps  only  one  musket  per  month.  However,  the 
major  arms  manufacturers  of  Europe  could  churn  out  battle- 
ready  weapons  by  the  thousands.  As  one  visitor  to  Britain's 
industrial  heartland  noted,  the  factories  of  Birmingham  alone 
produced  " a  prodigious  amount  for  exportation  ...  annually 
above  a  hundred  fifty  thousand  muskets''. 

But  France,  still  smarting  from  its  defeat  at  the  hands  of  its 
longtime  British  rivals  and  the  resultant  loss  of  Quebec  in  the 
Seven  Years'  War  little  more  than  a  decade  earlier,  saw  an 
opportunity  to  redress  the  balance  of  power  in  its  favor  by 
intervening  in  the  American  war.  Spain,  allied  dynastically  with 
France,  saw  a  ready-made  opportunity  to  secure  its  own  New 
World  possessions  against  Britain's  rising  power. 

Ferreiro  contends  that  the  primary  purpose  of  our  Declara¬ 
tion  of  Independence  in  1776  was  to  enable  America  to  secure 
French  and  Spanish  support  that  would  likely  not  have  been 
fully  forthcoming  in  a  mere  civil  war  between  the  Colonies  and 
Mother  England  over  the  specific  acts  of  Parliament.  And  the 
French,  for  their  part,  had  anticipated  such  a  revanchist  op¬ 
portunity  would  open  for  them  in  the  New  World  even  before 
the  Americans  fully  embraced  the  idea  of  independence. 

The  Founders  believed  they  could  not  expect  the  French  and 
Spanish  to  deal  with  them  as  a  sovereign  people  until  they 


From  the  Book’s  Introduction: 

"The  problem  was  that  the  new  American  nation  had 
begun  its  war  against  British  rule  stunningly  incapable  of 
fending  for  itself,  like  a  rebellious  adolescent  who  takes 
leave  of  his  family  without  a  penny  to  his  name.  It  had  no 
navy,  little  in  the  way  of  artillery,  and  a  ragtag  army  and 
militia  that  were  bereft  of  even  the  most  basic  ingredient 
of  modern  warfare:  gunpowder.  Soon  after  the  Battle  of 
Bunker  Hill,  Benjamin  Franklin  noted  that  'the  Army  had 
not  five  rounds  of  powder  a  man.  The  world  wondered 
that  we  so  seldom  fired  a  cannon;  we  could  not  afford  it.' 
America,  in  short,  desperately  needed  to  bring  France 
and  Spain  directly  into  the  war,  the  only  nations  powerful 
enough  to  carry  the  fight  straight  to  the  British  army  as  well 
as  to  bring  the  British  navy  into  a  wider  conflict  that  would 
draw  it  away  from  American  shores  and  sap  its  strength. 

"Both  France  and  Spain  had  allowed  clandestine  aid  to 
flow  to  the  Americans  since  before  the  fighting  started, 
but  this  was  proving  insufficient  for  the  scale  of  the 
conflict.  Neither  Louis  XVI  nor  Carlos  III  would  openly 
take  sides  in  a  British  civil  war;  America  had  to  demon¬ 
strate  that  it  was  an  independent  nation  fighting  against 
a  common  British  enemy.  The  document  that  emerged 
from  under  Jefferson's  hand,  clearly  stating  that  'these 
United  Colonies  are,  and  of  Right  ought  to  be  Free  and 
Independent  States,'  was  in  fact  an  engraved  invitation 
to  France  and  Spain  asking  them  to  go  to  war  alongside 
the  Americans." 


had  definitively  proclaimed  themselves  as  such.  John  Adams, 
no  fan  of  foreign  entanglements,  admitted,  "Foreign  powers 
could  not  be  expected  to  acknowledge  us,  till  we  had  acknowl¬ 
edged  ourselves  as  an  independent  nation."  And  Thomas  Jef¬ 
ferson  noted  that  "a  declaration  of  independence  alone  could 
allow  European  powers  to  treat  with  us."  It  also  is  Ferreiro's 
belief  that,  despite  the  significant  outlays  in  support  of  the 
Americans,  including  up  to  $50  billion  in  fiscal  aid  in  addition 
to  weapons  and  supplies,  our  War  of  Independence  was  not 
the  decisive  element  in  the  French  fiscal  crisis  that  led  to  their 
own  Revolution  shortly  after  ours.  That  crisis  had  long  been 
brewing  because  of  the  deep  indebtedness  of  the  French  state. 

Ferreiro's  book  was  a  finalist  in  contention  for  the  2017  Pulit¬ 
zer  Prize  in  history  and  is  an  excellent  read.  Congratulations, 
Dr.  Ferreiro. 

Ferreiro  is  a  naval  architect  and  historian  and  holds  a  Ph.D. 
from  the  Imperial  College  in  London,  England.  He  teaches 
history  at  George  Mason  University  and  has  written  and 
edited  several  books  on  naval  history.  He  also  is  research 
director  at  the  Defense  Acquisition  University  (DAU)  and 
executive  editor  of  Defense  AT&L's  sister  publication,  DAU's 
Defense  Acquisition  Research  Journal. 


This  article  was  compiled  by  the  editors  of  Defense  AT&L  magazine. 
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LETTER  TO  THE  EDITOR 


“Tools  for  Deciphering  Best  Value”  by  William  Sims  Curry  in 
the  May-June  2017  issue  of  Defense  AT&L  magazine  is  an  ex¬ 
ample  of  how,  given  enough  time,  viewpoints  swing  back  to 
those  held  long  before.  A  little  more  than  12  years  ago,  Defense 
AT&L  published  an  article  (in  two  parts)  in  the  September- 
October  and  November-December  2004  issues  on  Best  Value 
or  Trade-off  source  selection  methodology.  In  2004,  this  form 
of  source  selection  was  not  commonly  used  across  the  ac¬ 
quisition  community;  instead,  the  very  quantitative  scoring 
system  propounded  by  Mr.  Curry  was  much  more  the  norm. 

What  is  described  now  as  a  weakness  was,  in  2004,  described 
as  a  strength.  It  is  best  understood  that  no  one  single  process 
will  be  the  best  for  all  situations.  But  that  is  what  we  all  too 
often  try  to  do.  I  will  state  that  there  are  source  selections 
where  the  quantitative  methods  of  source  selection  may  be 
the  best,  and  the  particular  case  cited  by  the  article  appears 
(as  presented)  to  be  one  of  those  scenarios.  But,  for  a  very 
complicated  technical  evaluation,  I  would  suggest  that  quan¬ 
titative  methods  are  not  going  to  be  the  best  solution.  Several 
things  involved  lend  credibility  to  my  statement. 

One  is  that— even  just  talking  about  a  weapon  system's  tech¬ 
nical  side— if  everything  is  going  to  add  up  to  100  percent  for 
total  weighting,  one  of  two  things  will  be  the  case:  Either  no 
technical  requirement  will  have  a  very  high  weight  or  some  of 
the  technical  requirements  will  have  such  little  weighting  as 
to  make  us  ask  why  we  should  bother  to  evaluate  this.  And 
that  only  increases  when  you  add  the  nontechnical  areas  of 
management,  risk,  cost  and  past  performance. 

The  second  thing  is  that  requirements  interrelationships  are 
not  necessarily  static.  By  itself,  the  technical  requirement  may 
have  a  relative  importance  of  A,  B  and  C.  However,  if  you  are 
able  to  achieve  objective  values  of  all  three,  there  may  be 
a  synergistic  effect  and  the  total  overall  impact  of  all  three 
proves  to  be  greater  than  A+B+C. 

Now,  let's  consider  that  some  performance  values  may  be 
worth  a  higher  cost.  That  sort  of  messes  with  the  value  ac¬ 
corded  to  cost  and  price.  So,  there  was  a  reason  we  started 
switching  in  the  early  2000s  to  the  greater  use  of  non-quan- 
titative  best  value,  "squishy"  as  it  may  seem. 

One  last  point.  Let's  look  at  a  sentence  from  the  article's 
conclusion:  "DoD  would  benefit  from  use  of  the  TWS  [Total 
Weighted  Score]  contractor  selection  process  that  distinctly 
identifies  the  contractor  offering  the  best  value  proposal, 
simplifies  the  technical/cost  trade-off  decision,  and  inhibits 
procurement  corruption.''  Let's  particularly  look  at  the  last 


part:  ”...  and  inhibits  procurement  corruption.''  Let's  face  it: 
If  we  have  a  real  problem  with  procurement  corruption,  we 
have  some  fundamental  issues  that  no  single  source  selec¬ 
tion  method  will  cure.  There  is  a  way  around  any  single  sys¬ 
tem.  It's  just  a  matter  of  having  enough  motivation  to  find  it. 

Respectfully, 

Alexander  R.  Slate 

Project  Management  Professional 

(The  letter  writer  is  a  retired  Air  Force  civilian ,  with  more  than  30 
years  of  experience  supporting  acquisition  in  a  number  of  different 
capacities  [project  manager ;  test engineering ,  research  scientist , 
and  acquisition  facilitator].  He  previously  worked  for  the  Brooks  Air 
Force  Base  Center  for  Acquisition  Excellence  in  San  Antonio ,  Texas.) 

NOTE  FROM  THE  EDITOR 

Defense  AT&L  magazine  encourages  its  readers  to  com¬ 
ment  and  contribute  to  the  ongoing  discussion  in  these 
pages  of  defense  acquisition. 


MDAP/MAIS  Program 
Manager  Changes 


With  the  assistance  of  the  Office  of  the  Secretary  of 
Defense,  Defense  AT&L  magazine  publishes  the  names 
of  incoming  and  outgoing  program  managers  for  major 
defense  acquisition  programs  (MDAPs)  and  major  auto¬ 
mated  information  system  (MAIS)  programs.  Below  are 
listed  the  leadership  changes  reported  for  both  civilian  and 
military  program  managers  in  March  and  April  2017. 

Navy/Marine  Corps 

CAPT  John  Dougherty  relieved  CAPT  Jaime  Engdahl  as 

program  manager  for  Precision  Strike  Weapons  (PMA 
201)  on  March  23. 

CAPT  Paul  Mitchell  relieved  CAPT  Ralph  Lee  as  program 
manager  for  the  Strike  Planning  and  Execution  Systems 
Program  Office  (PMA  281)  on  March  3. 

Dr.  Regan  Campbell  relieved  CAPT  Daniel  Brintzinghoffer 

as  program  manager  for  the  Frigate  Program  Office  (PMS 
515)  on  April  7. 
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WRITERS'  GUIDELINES  IN  BRIEF 


Purpose 

Defense  AT&L  is  a  bimonthly  magazine  published  by  DAU  Press, 
Defense  Acquisition  University,  for  senior  military  personnel, 
civilians,  defense  contractors  and  defense  industry  professionals 
in  program  management  and  the  acquisition,  technology  and  lo¬ 
gistics  workforce. 

Submission  Procedures 

Submit  articles  by  e-mail  to  datl@dau.mil.  Submissions  must  include 
each  author's  name,  mailing  address,  office  phone  number,  e-mail 
address,  and  brief  biographical  statement.  Each  must  also  be  ac¬ 
companied  by  a  copyright  release.  For  each  article  submitted,  please 
include  three  to  four  keywords  that  can  be  used  to  facilitate  Web  and 
data  base  searches. 

Receipt  of  your  submission  will  be  acknowledged  in  5  working  days. 
You  will  be  notified  of  our  publication  decision  in  2  to  3  weeks.  All 
decisions  are  final. 


Length 

Articles  should  be  1,500-2,500  words. 

Format 

Send  submissions  via  e-mail  as  Microsoft  Word  attachments. 

Graphics 

Do  not  embed  photographs  or  charts  in  the  manuscript.  Digital  files 
of  photos  or  graphics  should  be  sent  as  e-mail  attachments.  Each 
figure  or  chart  must  be  saved  as  a  separate  file  in  the  original  soft¬ 
ware  format  in  which  it  was  created. 

TIF  or  JPEG  files  must  have  a  resolution  of  300  pixels  per  inch; 
enhanced  resolutions  are  not  acceptable;  and  images  downloaded 
from  the  Web  are  not  of  adequate  quality  for  reproduction.  De¬ 
tailed  tables  and  charts  are  not  accepted  for  publication  because 
they  will  be  illegible  when  reduced  to  fit  at  most  one-third  of  a 
magazine  page. 


Deadlines 

Note:  If  the  magazine  fills  up  before  the  author  deadline,  submissions 
are  considered  for  the  following  issue. 


Issue 

January-February 

March-April 

May-June 

July-August 

September-October 

November-December 


Author  Deadline 

1  October 
1  December 
1  February 
1  April 
1  June 
1  August 


Audience 

Defense  AT&L  readers  are  mainly  acquisition  professionals  serving 
in  career  positions  covered  by  the  Defense  Acquisition  Workforce 
Improvement  Act  (DAWIA)  or  industry  equivalent. 

Style 

Defense  AT&L  prints  feature  stories  focusing  on  real  people  and 
events.  The  magazine  seeks  articles  that  reflect  author  experiences 
in  and  thoughts  about  acquisition  rather  than  pages  of  researched 
information.  Articles  should  discuss  the  individual's  experience  with 
problems  and  solutions  in  acquisition,  contracting,  logistics,  or  pro¬ 
gram  management,  or  with  emerging  trends. 


The  magazine  does  not  print  academic  papers;  fact  sheets;  technical 
papers;  white  papers;  or  articles  with  footnotes,  endnotes,  or  refer¬ 
ences.  Manuscripts  meeting  any  of  those  criteria  are  more  suitable 
for  DAU's  journal,  Defense  Acquisition  Research  Journal  (ARJ). 


Defense  AT&L  does  not  reprint  from  other  publications.  Please  do  not 
submit  manuscripts  that  have  appeared  elsewhere.  Defense  AT&L 
does  not  publish  endorsements  of  products  for  sale. 


Right  to  Use  Illustrations 

Non-DoD  photos  and  graphics  are  printed  only  with  written  per¬ 
mission  from  the  source.  It  is  the  author's  responsibility  to  obtain 
and  submit  permission  with  the  article.  Do  not  include  any  clas¬ 
sified  information. 

Author  Information 

Contact  and  biographical  information  will  be  included  with  each 
article  selected  for  publication.  Please  include  the  following  infor¬ 
mation  with  your  submission:  name,  position  title,  department,  in¬ 
stitution,  address,  phone  number  and  e-mail  address.  Also,  please 
supply  a  short  biographical  statement,  not  to  exceed  25  words.  We 
do  not  print  author  bio  photographs. 

Copyright 

All  articles  require  a  signed  Work  of  the  U.S.  Government/Copy¬ 
right  Release  form,  available  at  https://www.dau.mil/library/ 
defense-atl/Lists/PageContent/Attachments/6/DATLcopyright- 
release_032217.pdf.  Fill  out,  sign,  scan  and  e-mail  it  to  datl@dau. 
mil  or  fax  it  to  703-805-2917,  Attn:  Defense  AT&L. 

Alternatively,  you  may  submit  a  written  release  from  the  major  com¬ 
mand  (normally  the  public  affairs  office)  indicating  the  author  is  re¬ 
leasing  the  article  to  Defense  AT&L  for  publication  without  restriction. 

The  Defense  Acquisition  University  does  not  accept  copy¬ 
righted  material  for  publication  in  Defense  AT&L.  Articles  will 
be  considered  only  if  they  are  unrestricted.  This  is  in  keep¬ 
ing  with  the  University's  policy  that  our  publications  be  fully 
accessible  to  the  public  without  restriction.  All  articles  are 
in  the  public  domain  and  posted  to  the  University's  website, 
https://www.dau.mil. 


http:  / /www.  dau.mil  /library /defense-atl  /p /Writers-Guidelines 


